[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:47 dist-test-slave-f7th krb5kdc[16680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:47 dist-test-slave-f7th krb5kdc[16680](info): set up 2 sockets
Aug 11 00:18:47 dist-test-slave-f7th krb5kdc[16680](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 00:18:48.784595 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:18:48 dist-test-slave-f7th krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:18:49.011384 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40475
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:40475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:49.396546 16702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:49.396916 16702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:49.397061 16702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:49.404994 16702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:49.405126 16702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:49.405217 16702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:49.405300 16702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:49.405350 16702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:49.416086 16702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:40475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:49.417747 16702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:49.419755 16702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:49.439754 16712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:49.439734 16711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:49.440526 16715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:49.441618 16702 server_base.cc:1034] running on GCE node
I20260811 00:18:49.442562 16702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:49.444348 16702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:49.445597 16702 hybrid_clock.cc:648] HybridClock initialized: now 1786407529445562 us; error 123 us; skew 500 ppm
I20260811 00:18:49.446002 16702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:49 dist-test-slave-f7th krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407529, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:49.449774 16702 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:18:49.451604 16702 webserver.cc:460] Webserver started at http://127.16.71.190:39093/ using document root <none> and password file <none>
I20260811 00:18:49.452431 16702 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:49.452554 16702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:49.452920 16702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:49.457098 16702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "6541c3d4ada648cdbb707718ad09fd72"
format_stamp: "Formatted at 2026-08-11 00:18:49 on dist-test-slave-f7th"
server_key: "be703ce8aba991b49744fae6f0d692e0"
server_key_iv: "0319200c0e60166e139b9633743e5102"
server_key_version: "encryptionkey@0"
I20260811 00:18:49.458369 16702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "6541c3d4ada648cdbb707718ad09fd72"
format_stamp: "Formatted at 2026-08-11 00:18:49 on dist-test-slave-f7th"
server_key: "be703ce8aba991b49744fae6f0d692e0"
server_key_iv: "0319200c0e60166e139b9633743e5102"
server_key_version: "encryptionkey@0"
I20260811 00:18:49.465826 16702 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 00:18:49.470666 16732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:49.472347 16702 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 00:18:49.472610 16702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "6541c3d4ada648cdbb707718ad09fd72"
format_stamp: "Formatted at 2026-08-11 00:18:49 on dist-test-slave-f7th"
server_key: "be703ce8aba991b49744fae6f0d692e0"
server_key_iv: "0319200c0e60166e139b9633743e5102"
server_key_version: "encryptionkey@0"
I20260811 00:18:49.472985 16702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:49.496321 16702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:49.501459 16702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:49.501756 16702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:49.524784 16702 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:40475
I20260811 00:18:49.524786 16817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:40475 every 8 connection(s)
I20260811 00:18:49.526624 16702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:49.531342 16820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:49.536852 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 16702
I20260811 00:18:49.537169 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:18:49.539146 16670 external_mini_cluster.cc:1428] Setting key 945a16c28183bb9ebd6ed0ccdafcb8ca
I20260811 00:18:49.543656 16820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72: Bootstrap starting.
I20260811 00:18:49.547658 16820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:49.548959 16820 log.cc:824] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:49.552291 16820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72: No bootstrap required, opened a new log
I20260811 00:18:49.563855 16820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6541c3d4ada648cdbb707718ad09fd72" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40475 } }
I20260811 00:18:49.564399 16820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:49.564560 16820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6541c3d4ada648cdbb707718ad09fd72, State: Initialized, Role: FOLLOWER
I20260811 00:18:49.565383 16820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6541c3d4ada648cdbb707718ad09fd72" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40475 } }
I20260811 00:18:49.565709 16820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:49.565882 16820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:49.566116 16820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:49.568976 16820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6541c3d4ada648cdbb707718ad09fd72" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40475 } }
I20260811 00:18:49.569660 16820 leader_election.cc:302] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6541c3d4ada648cdbb707718ad09fd72; no voters:
I20260811 00:18:49.570380 16820 leader_election.cc:288] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:49.570587 16825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:49.571853 16825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [term 1 LEADER]: Becoming Leader. State: Replica: 6541c3d4ada648cdbb707718ad09fd72, State: Running, Role: LEADER
I20260811 00:18:49.572570 16825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6541c3d4ada648cdbb707718ad09fd72" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40475 } }
I20260811 00:18:49.573066 16820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 00:18:49 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:49.579093 16827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6541c3d4ada648cdbb707718ad09fd72. Latest consensus state: current_term: 1 leader_uuid: "6541c3d4ada648cdbb707718ad09fd72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6541c3d4ada648cdbb707718ad09fd72" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40475 } } }
I20260811 00:18:49.579566 16827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:49.578984 16826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6541c3d4ada648cdbb707718ad09fd72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6541c3d4ada648cdbb707718ad09fd72" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40475 } } }
I20260811 00:18:49.580173 16826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:49.585700 16837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:49.589879 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:49.543514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36236 (local address 127.16.71.190:40475)
0811 00:18:49.544584 (+ 1070us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:49.544602 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:49.546135 (+ 1533us) server_negotiation.cc:407] Connection header received
0811 00:18:49.555507 (+ 9372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:49.555539 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:49.556151 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:49.556690 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:49.559941 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:49.562652 (+ 2711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:49.565181 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:49.566475 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:49.566772 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:49.575424 (+ 8652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:49.575439 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:49.575468 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:49.575541 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:49.581393 (+ 5852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:49.582856 (+ 1463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:49.582866 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:49.582876 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:49.583193 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:49.584282 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:49.584290 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:49.584294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:49.585584 (+ 1290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:49.586242 (+ 658us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:49.587422 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0811 00:18:49.587998 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":511,"threads_started":1}
I20260811 00:18:49.594184 16837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:49.605907 16837 catalog_manager.cc:1347] Generated new cluster ID: 6bce9b8756df4b8592ada43c44b19599
I20260811 00:18:49.606069 16837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:49.623723 16837 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:49.625574 16837 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:49.635789 16837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6541c3d4ada648cdbb707718ad09fd72: Generated new TSK 0
I20260811 00:18:49.636898 16837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:18:49.708129 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40475
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:50.118388 16856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:50.118717 16856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:50.118841 16856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:50.126850 16856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:50.127002 16856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:50.127166 16856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:18:50.137806 16856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40475
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:50.139261 16856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:50.140694 16856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:50.154927 16862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:50.154994 16868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:50.154963 16872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:50.155642 16856 server_base.cc:1034] running on GCE node
I20260811 00:18:50.156417 16856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:50.157513 16856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:50.158766 16856 hybrid_clock.cc:648] HybridClock initialized: now 1786407530158710 us; error 43 us; skew 500 ppm
I20260811 00:18:50.159133 16856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:50 dist-test-slave-f7th krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407530, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:50.162596 16856 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:18:50.164384 16856 webserver.cc:460] Webserver started at http://127.16.71.129:45983/ using document root <none> and password file <none>
I20260811 00:18:50.165187 16856 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:50.165311 16856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:50.165645 16856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:50.169035 16856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "0b5144b3cc46430787b038516831bf2b"
format_stamp: "Formatted at 2026-08-11 00:18:50 on dist-test-slave-f7th"
server_key: "52580de9afad99d1b9ee776919afa377"
server_key_iv: "241cbb5fc1a3bf0730d92f95545655cb"
server_key_version: "encryptionkey@0"
I20260811 00:18:50.170073 16856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "0b5144b3cc46430787b038516831bf2b"
format_stamp: "Formatted at 2026-08-11 00:18:50 on dist-test-slave-f7th"
server_key: "52580de9afad99d1b9ee776919afa377"
server_key_iv: "241cbb5fc1a3bf0730d92f95545655cb"
server_key_version: "encryptionkey@0"
I20260811 00:18:50.176311 16856 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:18:50.180734 16886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:50.182309 16856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:18:50.182487 16856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "0b5144b3cc46430787b038516831bf2b"
format_stamp: "Formatted at 2026-08-11 00:18:50 on dist-test-slave-f7th"
server_key: "52580de9afad99d1b9ee776919afa377"
server_key_iv: "241cbb5fc1a3bf0730d92f95545655cb"
server_key_version: "encryptionkey@0"
I20260811 00:18:50.182687 16856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:50.201546 16856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:50.206792 16856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:50.207237 16856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:50.208513 16856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:18:50.210294 16856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:18:50.210387 16856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:50.210500 16856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:18:50.210547 16856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:50.254884 16856 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:35163
I20260811 00:18:50.254961 17072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:35163 every 8 connection(s)
I20260811 00:18:50.256418 16856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:18:50.264671 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 16856
I20260811 00:18:50.264943 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:18:50.265782 16670 external_mini_cluster.cc:1428] Setting key 787227c38587b3fb93c45d433385895d
Aug 11 00:18:50 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407530, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:50.280049 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:50.259384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:51641 (local address 127.16.71.190:40475)
0811 00:18:50.259860 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:50.259866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:50.261042 (+ 1176us) server_negotiation.cc:407] Connection header received
0811 00:18:50.262966 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:50.262972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:50.263062 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:50.263162 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:50.264914 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.266068 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.268224 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.268682 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.268808 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:50.272715 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:50.272720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:50.272724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:50.272760 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:50.276035 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:50.277093 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:50.277100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:50.277103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:50.277291 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:50.277801 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:50.277807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:50.277811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:50.278210 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:50.278384 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:50.279381 (+ 997us) server_negotiation.cc:299] Negotiation successful
0811 00:18:50.279664 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":236,"threads_started":1}
I20260811 00:18:50.281330 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:50.259967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40475 (local address 127.16.71.129:51641)
0811 00:18:50.260823 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:50.260873 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:50.262656 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:50.263442 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:50.263451 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:50.264020 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:50.264690 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:50.264715 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.266292 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.266297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:50.268068 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:50.268085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.268947 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.268956 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:50.269207 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:50.270007 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:50.270034 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:50.272418 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:50.276240 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:50.276250 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:50.276267 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:50.276959 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:50.277416 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:50.277422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:50.277424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:50.277655 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:50.278353 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:50.278360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:50.278826 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 00:18:50.279168 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 00:18:50.279746 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":350,"threads_started":1}
I20260811 00:18:50.287909 17073 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40475
I20260811 00:18:50.288441 17073 heartbeater.cc:464] Registering TS with master...
I20260811 00:18:50.289636 17073 heartbeater.cc:511] Master 127.16.71.190:40475 requested a full tablet report, sending...
I20260811 00:18:50.292842 16762 ts_manager.cc:194] Registered new tserver with Master: 0b5144b3cc46430787b038516831bf2b (127.16.71.129:35163)
I20260811 00:18:50.295162 16762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:51641
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:18:50.329223 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40475
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:50.710518 17082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:50.710860 17082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:50.710968 17082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:50.718971 17082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:50.719108 17082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:50.719238 17082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:18:50.729980 17082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40475
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:50.731374 17082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:50.733002 17082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:50.750579 17099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:50.752492 17101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:50.752599 17098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:50.752594 17082 server_base.cc:1034] running on GCE node
I20260811 00:18:50.753383 17082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:50.755028 17082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:50.756361 17082 hybrid_clock.cc:648] HybridClock initialized: now 1786407530756284 us; error 45 us; skew 500 ppm
I20260811 00:18:50.756784 17082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:50 dist-test-slave-f7th krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407530, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:50.760156 17082 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:18:50.761906 17082 webserver.cc:460] Webserver started at http://127.16.71.130:35689/ using document root <none> and password file <none>
I20260811 00:18:50.762655 17082 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:50.762769 17082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:50.763079 17082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:50.766369 17082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "2ab4355209ad41148e1614e96c707f76"
format_stamp: "Formatted at 2026-08-11 00:18:50 on dist-test-slave-f7th"
server_key: "6e77e9fa9034c6dc07f4c58eea8854c8"
server_key_iv: "4fb278ec087b97eb4388ce6a9032954c"
server_key_version: "encryptionkey@0"
I20260811 00:18:50.767325 17082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "2ab4355209ad41148e1614e96c707f76"
format_stamp: "Formatted at 2026-08-11 00:18:50 on dist-test-slave-f7th"
server_key: "6e77e9fa9034c6dc07f4c58eea8854c8"
server_key_iv: "4fb278ec087b97eb4388ce6a9032954c"
server_key_version: "encryptionkey@0"
I20260811 00:18:50.774720 17082 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260811 00:18:50.779193 17116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:50.781005 17082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:18:50.781190 17082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "2ab4355209ad41148e1614e96c707f76"
format_stamp: "Formatted at 2026-08-11 00:18:50 on dist-test-slave-f7th"
server_key: "6e77e9fa9034c6dc07f4c58eea8854c8"
server_key_iv: "4fb278ec087b97eb4388ce6a9032954c"
server_key_version: "encryptionkey@0"
I20260811 00:18:50.781392 17082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:50.806365 17082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:50.811559 17082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:50.811867 17082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:50.813108 17082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:18:50.814707 17082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:18:50.814803 17082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:50.814944 17082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:18:50.814992 17082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:50.858182 17082 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:36409
I20260811 00:18:50.858328 17310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:36409 every 8 connection(s)
I20260811 00:18:50.859618 17082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:18:50.861896 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17082
I20260811 00:18:50.862093 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:18:50.862599 16670 external_mini_cluster.cc:1428] Setting key 445dc3d0ba1eecf62ddeefa4c0a27ee2
Aug 11 00:18:50 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407530, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:50.885747 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:50.862685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:59367 (local address 127.16.71.190:40475)
0811 00:18:50.863190 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:50.863198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:50.864440 (+ 1242us) server_negotiation.cc:407] Connection header received
0811 00:18:50.867300 (+ 2860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:50.867306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:50.867392 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:50.867503 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:50.870112 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.870992 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.873941 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.874565 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.874742 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:50.879209 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:50.879214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:50.879218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:50.879259 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:50.882188 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:50.883095 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:50.883100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:50.883102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:50.883252 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:50.883758 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:50.883762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:50.883764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:50.884078 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:50.884228 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:50.885111 (+ 883us) server_negotiation.cc:299] Negotiation successful
0811 00:18:50.885378 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":240,"threads_started":1}
I20260811 00:18:50.887058 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:50.863376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40475 (local address 127.16.71.130:59367)
0811 00:18:50.864273 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:50.864337 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:50.866854 (+ 2517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:50.867861 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:50.867876 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:50.868819 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:50.869815 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:50.869853 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.871227 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.871234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:50.873664 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:50.873687 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:50.874886 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:50.874896 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:50.875156 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:50.876024 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:50.876052 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:50.878902 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:50.882392 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:50.882401 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:50.882419 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:50.882937 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:50.883391 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:50.883396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:50.883398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:50.883627 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:50.884223 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:50.884233 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:50.884681 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 00:18:50.884966 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 00:18:50.885588 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":336,"threads_started":1}
I20260811 00:18:50.889037 17311 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40475
I20260811 00:18:50.889438 17311 heartbeater.cc:464] Registering TS with master...
I20260811 00:18:50.890272 17311 heartbeater.cc:511] Master 127.16.71.190:40475 requested a full tablet report, sending...
I20260811 00:18:50.892184 16762 ts_manager.cc:194] Registered new tserver with Master: 2ab4355209ad41148e1614e96c707f76 (127.16.71.130:36409)
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:18:50.893165 16762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:59367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:18:50.925503 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40475
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:18:51.299553 17073 heartbeater.cc:503] Master 127.16.71.190:40475 was elected leader, sending a full tablet report...
W20260811 00:18:51.309461 17317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:51.309813 17317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:51.309967 17317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:51.317828 17317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:51.317987 17317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:51.318151 17317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:18:51.328861 17317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40475
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:51.330353 17317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:51.331862 17317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:51.347136 17329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:51.347065 17330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:51.347518 17333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:51.348773 17317 server_base.cc:1034] running on GCE node
I20260811 00:18:51.349496 17317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:51.350553 17317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:51.351794 17317 hybrid_clock.cc:648] HybridClock initialized: now 1786407531351724 us; error 60 us; skew 500 ppm
I20260811 00:18:51.352172 17317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:51.355796 17317 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:18:51.357554 17317 webserver.cc:460] Webserver started at http://127.16.71.131:40781/ using document root <none> and password file <none>
I20260811 00:18:51.358261 17317 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:51.358412 17317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:51.358716 17317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:51.362150 17317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "5fc3555c2e8449398ca7ae90da06f7b4"
format_stamp: "Formatted at 2026-08-11 00:18:51 on dist-test-slave-f7th"
server_key: "9e85cc9270722521903bd5862bc7cae8"
server_key_iv: "a096f0a2b30c82c205a2e3e819424366"
server_key_version: "encryptionkey@0"
I20260811 00:18:51.363111 17317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "5fc3555c2e8449398ca7ae90da06f7b4"
format_stamp: "Formatted at 2026-08-11 00:18:51 on dist-test-slave-f7th"
server_key: "9e85cc9270722521903bd5862bc7cae8"
server_key_iv: "a096f0a2b30c82c205a2e3e819424366"
server_key_version: "encryptionkey@0"
I20260811 00:18:51.369282 17317 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:18:51.373515 17344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:51.375084 17317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:18:51.375280 17317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "5fc3555c2e8449398ca7ae90da06f7b4"
format_stamp: "Formatted at 2026-08-11 00:18:51 on dist-test-slave-f7th"
server_key: "9e85cc9270722521903bd5862bc7cae8"
server_key_iv: "a096f0a2b30c82c205a2e3e819424366"
server_key_version: "encryptionkey@0"
I20260811 00:18:51.375500 17317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:51.401209 17317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:51.406199 17317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:51.406518 17317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:51.407719 17317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:18:51.409358 17317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:18:51.409449 17317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:51.409575 17317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:18:51.409626 17317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:51.448429 17317 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:41431
I20260811 00:18:51.448495 17537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:41431 every 8 connection(s)
I20260811 00:18:51.449935 17317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:18:51.450490 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17317
I20260811 00:18:51.450686 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:18:51.451264 16670 external_mini_cluster.cc:1428] Setting key b4afe6b85a580f0bba11ffac01ede0c2
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:51.473486 17541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:51.453037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:40413 (local address 127.16.71.190:40475)
0811 00:18:51.453617 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:51.453622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:51.454783 (+ 1161us) server_negotiation.cc:407] Connection header received
0811 00:18:51.456947 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:51.456952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:51.457030 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:51.457128 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:51.458866 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.459812 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.461629 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.462075 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.462181 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:51.465996 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:51.466004 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:51.466009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:51.466056 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:51.469519 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.470630 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.470635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.470638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.470801 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.471403 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.471408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.471410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.471734 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:51.471912 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:51.472869 (+ 957us) server_negotiation.cc:299] Negotiation successful
0811 00:18:51.473088 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":328,"threads_started":1}
I20260811 00:18:51.474579 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:51.453731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40475 (local address 127.16.71.131:40413)
0811 00:18:51.454613 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:51.454662 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:51.456609 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:51.457354 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:51.457363 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:51.458021 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:51.458669 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:51.458689 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.460014 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.460023 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:51.461456 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:51.461471 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.462276 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.462281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:51.462497 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:51.463184 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:51.463210 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:51.465660 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:51.469719 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:51.469729 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:51.469745 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:51.470438 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:51.471004 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:51.471008 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:51.471010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:51.471255 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:51.471951 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:51.471960 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:51.472351 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 00:18:51.472678 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 00:18:51.473260 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":341,"threads_started":1}
I20260811 00:18:51.476286 17538 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40475
I20260811 00:18:51.476675 17538 heartbeater.cc:464] Registering TS with master...
I20260811 00:18:51.477541 17538 heartbeater.cc:511] Master 127.16.71.190:40475 requested a full tablet report, sending...
I20260811 00:18:51.479357 16762 ts_manager.cc:194] Registered new tserver with Master: 5fc3555c2e8449398ca7ae90da06f7b4 (127.16.71.131:41431)
I20260811 00:18:51.480396 16762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:40413
I20260811 00:18:51.481251 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:18:51.548265 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:51.529143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35374 (local address 127.16.71.129:35163)
0811 00:18:51.529924 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:51.529935 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:51.530612 (+ 677us) server_negotiation.cc:407] Connection header received
0811 00:18:51.530790 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:51.530798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:51.531078 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:51.531371 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:51.533463 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.534645 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.536276 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.537061 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.537188 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:51.541742 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:51.541760 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:51.541779 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:51.541838 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:51.544835 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.545716 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.545726 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.545728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.545908 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.546387 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.546392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.546394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.546850 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:51.547128 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:51.547565 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 00:18:51.547851 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":339,"threads_started":1}
W20260811 00:18:51.549772 16945 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35374
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:18:51.609231 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:51.593457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.16.71.129:35163)
0811 00:18:51.593641 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:51.593649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:51.594026 (+ 377us) server_negotiation.cc:407] Connection header received
0811 00:18:51.594218 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:51.594223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:51.594295 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:51.594389 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:51.595869 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.596609 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.598023 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.598515 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.598643 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:51.603260 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:51.603267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:51.603271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:51.603320 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:51.606167 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.607026 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.607031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.607034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.607202 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.607627 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.607631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.607634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.607915 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:51.608164 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:51.608533 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 00:18:51.608915 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:18:51.611075 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 16856
I20260811 00:18:51.626672 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:51.612205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.16.71.129:35163)
0811 00:18:51.612415 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:51.612422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:51.612836 (+ 414us) server_negotiation.cc:407] Connection header received
0811 00:18:51.613000 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:51.613004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:51.613071 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:51.613149 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:51.614554 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.615297 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.616841 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.617321 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.617441 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:51.619846 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:51.619852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:51.619855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:51.619895 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:51.623080 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.624151 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.624164 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.624167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.624376 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.624992 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.624998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.625001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.625331 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:51.625570 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:51.626015 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 00:18:51.626242 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:18:51.627593 17065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:51.713609 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 16856
I20260811 00:18:51.741529 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17082
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:18:51.758993 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:51.742605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.16.71.130:36409)
0811 00:18:51.743175 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:51.743182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:51.743208 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:18:51.743270 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:51.743276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:51.743537 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:51.743723 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:51.745019 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.746114 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.747459 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.748078 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.748190 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:51.752568 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:51.752585 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:51.752603 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:51.752702 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:51.755733 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.756514 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.756522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.756524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.756813 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.757269 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.757273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.757275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.757667 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:51.757982 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:51.758343 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 00:18:51.758642 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":271,"threads_started":1}
I20260811 00:18:51.760396 17301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:51.847275 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17082
I20260811 00:18:51.868180 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17317
Aug 11 00:18:51 dist-test-slave-f7th krb5kdc[16680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:18:51.887519 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:51.869465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.16.71.131:41431)
0811 00:18:51.870177 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:51.870185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:51.870216 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:18:51.870279 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:51.870285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:51.870515 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:51.870682 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:51.872380 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.873571 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.875014 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:51.875666 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:51.875808 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:51.880531 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:51.880550 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:51.880568 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:51.880891 (+ 323us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:51.883920 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.885063 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.885072 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.885074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.885249 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:51.885692 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:51.885696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:51.885698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:51.886126 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:51.886490 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:51.886793 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 00:18:51.887123 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":368,"threads_started":1}
I20260811 00:18:51.889050 17533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:51.973965 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17317
I20260811 00:18:52.000566 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 16702
I20260811 00:18:52.002194 16810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:52.109198 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 16702
2026-08-11T00:18:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4715 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:52 dist-test-slave-f7th krb5kdc[17577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:52 dist-test-slave-f7th krb5kdc[17577](info): set up 2 sockets
Aug 11 00:18:52 dist-test-slave-f7th krb5kdc[17577](info): commencing operation
krb5kdc: starting...
W20260811 00:18:53.393287 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:18:53 dist-test-slave-f7th krb5kdc[17577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:18:53.613248 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39641
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:39641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:53.963527 17608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:53.963907 17608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:53.964057 17608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:53.971787 17608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:53.971956 17608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:53.972048 17608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:53.972146 17608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:53.972215 17608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:53.982777 17608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:39641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:53.984313 17608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:53.985788 17608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:53.998780 17624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:53.998768 17622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:53.998931 17621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:53.999701 17608 server_base.cc:1034] running on GCE node
I20260811 00:18:54.000679 17608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:54.002447 17608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:54.003747 17608 hybrid_clock.cc:648] HybridClock initialized: now 1786407534003676 us; error 67 us; skew 500 ppm
I20260811 00:18:54.004196 17608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:54 dist-test-slave-f7th krb5kdc[17577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407534, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:54.011294 17608 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:18:54.013273 17608 webserver.cc:460] Webserver started at http://127.16.71.190:34423/ using document root <none> and password file <none>
I20260811 00:18:54.014209 17608 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:54.014348 17608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:54.014724 17608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:54.018285 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "b84c2264ad95499990d1ba739f2a5661"
format_stamp: "Formatted at 2026-08-11 00:18:54 on dist-test-slave-f7th"
server_key: "fdc4495076f6bb5f10bfdc88690c0cb1"
server_key_iv: "6b58440447a4ce73dac9db8236946d91"
server_key_version: "encryptionkey@0"
I20260811 00:18:54.019160 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "b84c2264ad95499990d1ba739f2a5661"
format_stamp: "Formatted at 2026-08-11 00:18:54 on dist-test-slave-f7th"
server_key: "fdc4495076f6bb5f10bfdc88690c0cb1"
server_key_iv: "6b58440447a4ce73dac9db8236946d91"
server_key_version: "encryptionkey@0"
I20260811 00:18:54.025413 17608 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:18:54.029739 17635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:54.031307 17608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.004s
I20260811 00:18:54.031518 17608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "b84c2264ad95499990d1ba739f2a5661"
format_stamp: "Formatted at 2026-08-11 00:18:54 on dist-test-slave-f7th"
server_key: "fdc4495076f6bb5f10bfdc88690c0cb1"
server_key_iv: "6b58440447a4ce73dac9db8236946d91"
server_key_version: "encryptionkey@0"
I20260811 00:18:54.031744 17608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:54.070600 17608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:54.075271 17608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:54.075580 17608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:54.094399 17608 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:39641
I20260811 00:18:54.094431 17725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:39641 every 8 connection(s)
I20260811 00:18:54.095911 17608 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:54.100067 17726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:54.106010 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17608
I20260811 00:18:54.106251 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:18:54.106870 16670 external_mini_cluster.cc:1428] Setting key d7ee637a5cdc91753a95f6a24326269b
I20260811 00:18:54.112864 17726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661: Bootstrap starting.
I20260811 00:18:54.117074 17726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:54.118547 17726 log.cc:824] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:54.122561 17726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661: No bootstrap required, opened a new log
Aug 11 00:18:54 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:54.131665 17726 raft_consensus.cc:348] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84c2264ad95499990d1ba739f2a5661" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39641 } }
I20260811 00:18:54.132205 17726 raft_consensus.cc:374] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:54.132328 17726 raft_consensus.cc:729] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84c2264ad95499990d1ba739f2a5661, State: Initialized, Role: FOLLOWER
I20260811 00:18:54.133009 17726 consensus_queue.cc:260] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84c2264ad95499990d1ba739f2a5661" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39641 } }
I20260811 00:18:54.133442 17726 raft_consensus.cc:388] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:54.133608 17726 raft_consensus.cc:482] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:54.133738 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:54.109077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.16.71.190:39641)
0811 00:18:54.110085 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:54.110105 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:54.110168 (+ 63us) server_negotiation.cc:407] Connection header received
0811 00:18:54.112276 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:54.112306 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:54.112806 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:54.113210 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:54.115229 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:54.116763 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:54.118644 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:54.119755 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:54.119991 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:54.125028 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:54.125039 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:54.125060 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:54.125115 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:54.128454 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:54.129485 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:54.129493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:54.129504 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:54.129707 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:54.130153 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:54.130157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:54.130162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:54.130829 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:54.131127 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:54.131676 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 00:18:54.132204 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":421,"threads_started":1}
I20260811 00:18:54.133821 17726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:54.136482 17726 raft_consensus.cc:504] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84c2264ad95499990d1ba739f2a5661" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39641 } }
I20260811 00:18:54.137166 17726 leader_election.cc:302] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b84c2264ad95499990d1ba739f2a5661; no voters:
I20260811 00:18:54.137823 17726 leader_election.cc:288] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:54.137974 17742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:54.138685 17742 raft_consensus.cc:686] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [term 1 LEADER]: Becoming Leader. State: Replica: b84c2264ad95499990d1ba739f2a5661, State: Running, Role: LEADER
I20260811 00:18:54.139233 17742 consensus_queue.cc:237] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84c2264ad95499990d1ba739f2a5661" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39641 } }
I20260811 00:18:54.139925 17726 sys_catalog.cc:564] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:54.141935 17743 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b84c2264ad95499990d1ba739f2a5661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84c2264ad95499990d1ba739f2a5661" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39641 } } }
I20260811 00:18:54.142933 17743 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:54.142549 17746 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b84c2264ad95499990d1ba739f2a5661. Latest consensus state: current_term: 1 leader_uuid: "b84c2264ad95499990d1ba739f2a5661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84c2264ad95499990d1ba739f2a5661" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39641 } } }
I20260811 00:18:54.143455 17746 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:54.144953 17757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:54.150616 17757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:54.160199 17757 catalog_manager.cc:1347] Generated new cluster ID: b4e328cd12904615a6082ab4c63c51a9
I20260811 00:18:54.160322 17757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:54.170398 17757 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:54.172056 17757 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:54.180061 17757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b84c2264ad95499990d1ba739f2a5661: Generated new TSK 0
I20260811 00:18:54.181079 17757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:18:54.248574 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39641
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:54.638288 17783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:54.638608 17783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:54.638732 17783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:54.646672 17783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:54.646806 17783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:54.646989 17783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:18:54.657416 17783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:54.658808 17783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:54.660212 17783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:54.674870 17796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:54.674880 17799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:54.674839 17795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:54.675953 17783 server_base.cc:1034] running on GCE node
I20260811 00:18:54.676813 17783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:54.677959 17783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:54.679234 17783 hybrid_clock.cc:648] HybridClock initialized: now 1786407534679152 us; error 69 us; skew 500 ppm
I20260811 00:18:54.679613 17783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:54 dist-test-slave-f7th krb5kdc[17577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407534, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:54.683400 17783 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:18:54.685091 17783 webserver.cc:460] Webserver started at http://127.16.71.129:45847/ using document root <none> and password file <none>
I20260811 00:18:54.685863 17783 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:54.685992 17783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:54.686327 17783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:54.690121 17783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "c96ee2ca851949c080e291a192bd2c5a"
format_stamp: "Formatted at 2026-08-11 00:18:54 on dist-test-slave-f7th"
server_key: "3c7da0765be80279d61b37e8e2169f38"
server_key_iv: "c380b53de59ae399db2308c8c71d0dec"
server_key_version: "encryptionkey@0"
I20260811 00:18:54.691040 17783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "c96ee2ca851949c080e291a192bd2c5a"
format_stamp: "Formatted at 2026-08-11 00:18:54 on dist-test-slave-f7th"
server_key: "3c7da0765be80279d61b37e8e2169f38"
server_key_iv: "c380b53de59ae399db2308c8c71d0dec"
server_key_version: "encryptionkey@0"
I20260811 00:18:54.697332 17783 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:18:54.701699 17815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:54.703275 17783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 00:18:54.703480 17783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "c96ee2ca851949c080e291a192bd2c5a"
format_stamp: "Formatted at 2026-08-11 00:18:54 on dist-test-slave-f7th"
server_key: "3c7da0765be80279d61b37e8e2169f38"
server_key_iv: "c380b53de59ae399db2308c8c71d0dec"
server_key_version: "encryptionkey@0"
I20260811 00:18:54.703825 17783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:54.728307 17783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:54.733121 17783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:54.733428 17783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:54.734581 17783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:18:54.736160 17783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:18:54.736243 17783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:54.736348 17783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:18:54.736404 17783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:54.769644 17783 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:40379
I20260811 00:18:54.769682 18002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:40379 every 8 connection(s)
I20260811 00:18:54.771104 17783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:18:54.774688 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17783
I20260811 00:18:54.774938 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:18:54.775831 16670 external_mini_cluster.cc:1428] Setting key 16578a5c71c22853fc311dc2c83cb512
Aug 11 00:18:54 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407534, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:54.796152 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:54.774098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:33289 (local address 127.16.71.190:39641)
0811 00:18:54.775307 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:54.775313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:54.776286 (+ 973us) server_negotiation.cc:407] Connection header received
0811 00:18:54.778748 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:54.778754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:54.778845 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:54.778946 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:54.780816 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:54.781573 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:54.783319 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:54.783763 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:54.783899 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:54.788235 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:54.788242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:54.788246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:54.788292 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:54.791202 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:54.792703 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:54.792711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:54.792714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:54.792924 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:54.793636 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:54.793641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:54.793643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:54.794085 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:54.794327 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:54.795567 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0811 00:18:54.795811 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":972,"threads_started":1}
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
I20260811 00:18:54.797706 18010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:54.774784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39641 (local address 127.16.71.129:33289)
0811 00:18:54.776067 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:54.776126 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:54.778362 (+ 2236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:54.779263 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:54.779273 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:54.779888 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:54.780559 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:54.780583 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:54.781770 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:54.781775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:54.783118 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:54.783134 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:54.784055 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:54.784063 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:54.784319 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:54.785092 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:54.785119 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:54.787840 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:54.791465 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:54.791478 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:54.791502 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:54.792419 (+ 917us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:54.793084 (+ 665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:54.793091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:54.793095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:54.793443 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:54.794314 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:54.794325 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:54.794931 (+ 606us) client_negotiation.cc:769] Sending connection context
0811 00:18:54.795363 (+ 432us) client_negotiation.cc:240] Negotiation successful
0811 00:18:54.796117 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":590,"threads_started":1}
I20260811 00:18:54.799809 18006 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
I20260811 00:18:54.800168 18006 heartbeater.cc:464] Registering TS with master...
I20260811 00:18:54.801198 18006 heartbeater.cc:511] Master 127.16.71.190:39641 requested a full tablet report, sending...
I20260811 00:18:54.803949 17662 ts_manager.cc:194] Registered new tserver with Master: c96ee2ca851949c080e291a192bd2c5a (127.16.71.129:40379)
I20260811 00:18:54.805661 17662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:33289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:18:54.828321 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39641
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:55.183109 18016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:55.183454 18016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:55.183559 18016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:55.191260 18016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:55.191407 18016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:55.191540 18016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:18:55.202340 18016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:55.203706 18016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:55.205217 18016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:55.220329 18025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:55.220407 18026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:55.220367 18028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:55.221294 18016 server_base.cc:1034] running on GCE node
I20260811 00:18:55.221948 18016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:55.222971 18016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:55.224211 18016 hybrid_clock.cc:648] HybridClock initialized: now 1786407535224146 us; error 61 us; skew 500 ppm
I20260811 00:18:55.224577 18016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:55 dist-test-slave-f7th krb5kdc[17577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407535, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:55.228104 18016 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:18:55.230181 18016 webserver.cc:460] Webserver started at http://127.16.71.130:35231/ using document root <none> and password file <none>
I20260811 00:18:55.230991 18016 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:55.231139 18016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:55.231477 18016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:55.234817 18016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "27ad9d47853c444c9d02c07bc4273c5e"
format_stamp: "Formatted at 2026-08-11 00:18:55 on dist-test-slave-f7th"
server_key: "15fcae0f5d85284c1ed717dfa497419c"
server_key_iv: "7cc94c2eaeacaa090d6d8d824f343589"
server_key_version: "encryptionkey@0"
I20260811 00:18:55.235853 18016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "27ad9d47853c444c9d02c07bc4273c5e"
format_stamp: "Formatted at 2026-08-11 00:18:55 on dist-test-slave-f7th"
server_key: "15fcae0f5d85284c1ed717dfa497419c"
server_key_iv: "7cc94c2eaeacaa090d6d8d824f343589"
server_key_version: "encryptionkey@0"
I20260811 00:18:55.242128 18016 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 00:18:55.246559 18037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:55.248860 18016 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:18:55.249123 18016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "27ad9d47853c444c9d02c07bc4273c5e"
format_stamp: "Formatted at 2026-08-11 00:18:55 on dist-test-slave-f7th"
server_key: "15fcae0f5d85284c1ed717dfa497419c"
server_key_iv: "7cc94c2eaeacaa090d6d8d824f343589"
server_key_version: "encryptionkey@0"
I20260811 00:18:55.249415 18016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:55.275772 18016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:55.281284 18016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:55.281615 18016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:55.282778 18016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:18:55.284395 18016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:18:55.284489 18016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:55.284648 18016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:18:55.284718 18016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 00:18:55.318694 18016 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:39535
I20260811 00:18:55.318745 18234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:39535 every 8 connection(s)
I20260811 00:18:55.320102 18016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:18:55.321054 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18016
I20260811 00:18:55.321283 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:18:55.321781 16670 external_mini_cluster.cc:1428] Setting key 3fd6842577af026634fd3df58ebd6bb6
Aug 11 00:18:55 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407535, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:18:55.342782 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.323229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:52915 (local address 127.16.71.190:39641)
0811 00:18:55.323667 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:55.323673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:55.325169 (+ 1496us) server_negotiation.cc:407] Connection header received
0811 00:18:55.327191 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:55.327197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:55.327288 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:55.327422 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:55.329159 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.330155 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.331814 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.332255 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.332383 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.336440 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:55.336446 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:55.336449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:55.336518 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:55.339248 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:55.340176 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:55.340181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:55.340183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:55.340333 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:55.340864 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:55.340869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:55.340872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:55.341223 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:55.341369 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:55.342152 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 00:18:55.342396 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":226,"threads_started":1}
I20260811 00:18:55.343930 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.324035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39641 (local address 127.16.71.130:52915)
0811 00:18:55.324973 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:55.325036 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:55.326878 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:55.327663 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:55.327672 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:55.328262 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:55.328958 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.328983 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.330341 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.330346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.331646 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.331660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.332508 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.332516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.332812 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.333513 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:55.333538 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:55.336154 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:55.339434 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:55.339445 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:55.339465 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:55.340037 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:55.340451 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:55.340457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:55.340461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:55.340724 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:55.341357 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:55.341366 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:55.341754 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 00:18:55.342061 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 00:18:55.342647 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":354,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
I20260811 00:18:55.345762 18235 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39641
I20260811 00:18:55.346089 18235 heartbeater.cc:464] Registering TS with master...
I20260811 00:18:55.346855 18235 heartbeater.cc:511] Master 127.16.71.190:39641 requested a full tablet report, sending...
I20260811 00:18:55.348435 17662 ts_manager.cc:194] Registered new tserver with Master: 27ad9d47853c444c9d02c07bc4273c5e (127.16.71.130:39535)
I20260811 00:18:55.349321 17662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:52915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:18:55.371598 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39641
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:55.723425 18251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:55.723747 18251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:55.723850 18251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:55.731679 18251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:55.731868 18251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:55.732034 18251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:18:55.742751 18251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:55.744122 18251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:55.745715 18251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:55.769210 18263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:55.769289 18266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:55.773044 18258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:55.775889 18251 server_base.cc:1034] running on GCE node
I20260811 00:18:55.776829 18251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:55.777987 18251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:55.779263 18251 hybrid_clock.cc:648] HybridClock initialized: now 1786407535779198 us; error 73 us; skew 500 ppm
I20260811 00:18:55.779672 18251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:55 dist-test-slave-f7th krb5kdc[17577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407535, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:55.783151 18251 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:18:55.784920 18251 webserver.cc:460] Webserver started at http://127.16.71.131:46247/ using document root <none> and password file <none>
I20260811 00:18:55.785647 18251 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:55.785759 18251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:55.786052 18251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:55.789343 18251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "8e790a75374d482d972aa097097ce50f"
format_stamp: "Formatted at 2026-08-11 00:18:55 on dist-test-slave-f7th"
server_key: "30e5bbfbbaed1b08bff91f7df0461089"
server_key_iv: "f91a062372cc0ce886194d8500f73e3a"
server_key_version: "encryptionkey@0"
I20260811 00:18:55.790203 18251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "8e790a75374d482d972aa097097ce50f"
format_stamp: "Formatted at 2026-08-11 00:18:55 on dist-test-slave-f7th"
server_key: "30e5bbfbbaed1b08bff91f7df0461089"
server_key_iv: "f91a062372cc0ce886194d8500f73e3a"
server_key_version: "encryptionkey@0"
I20260811 00:18:55.796241 18251 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:18:55.800239 18277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:55.801819 18251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:18:55.801996 18251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "8e790a75374d482d972aa097097ce50f"
format_stamp: "Formatted at 2026-08-11 00:18:55 on dist-test-slave-f7th"
server_key: "30e5bbfbbaed1b08bff91f7df0461089"
server_key_iv: "f91a062372cc0ce886194d8500f73e3a"
server_key_version: "encryptionkey@0"
I20260811 00:18:55.802205 18251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:55.809691 18006 heartbeater.cc:503] Master 127.16.71.190:39641 was elected leader, sending a full tablet report...
I20260811 00:18:55.825217 18251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:55.830068 18251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:55.830379 18251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:55.831487 18251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:18:55.833051 18251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:18:55.833139 18251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:55.833269 18251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:18:55.833330 18251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:55.867759 18251 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:36389
I20260811 00:18:55.867801 18467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:36389 every 8 connection(s)
I20260811 00:18:55.869284 18251 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:18:55.874970 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18251
I20260811 00:18:55.875157 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:18:55.875590 16670 external_mini_cluster.cc:1428] Setting key 1acf91d190c7312295d33557da6c3aa3
Aug 11 00:18:55 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407535, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:55.892194 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.872280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:53937 (local address 127.16.71.190:39641)
0811 00:18:55.872813 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:55.872818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:55.873874 (+ 1056us) server_negotiation.cc:407] Connection header received
0811 00:18:55.875955 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:55.875962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:55.876064 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:55.876164 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:55.878070 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.879005 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.880758 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.881187 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.881293 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.885411 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:55.885417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:55.885422 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:55.885466 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:55.888319 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:55.889467 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:55.889472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:55.889474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:55.889665 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:55.890241 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:55.890246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:55.890249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:55.890573 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:55.890730 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:55.891535 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 00:18:55.891790 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":248,"threads_started":1}
I20260811 00:18:55.893304 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.872926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39641 (local address 127.16.71.131:53937)
0811 00:18:55.873704 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:55.873745 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:55.875643 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:55.876444 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:55.876454 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:55.877170 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:55.877869 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.877899 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.879193 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.879198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.880547 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.880564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.881510 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.881516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.881766 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.882542 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:55.882570 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:55.885110 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:55.888537 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:55.888548 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:55.888563 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:55.889229 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:55.889796 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:55.889800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:55.889803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:55.890102 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:55.890729 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:55.890738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:55.891135 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 00:18:55.891409 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 00:18:55.892007 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":291,"threads_started":1}
I20260811 00:18:55.894881 18469 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39641
I20260811 00:18:55.895256 18469 heartbeater.cc:464] Registering TS with master...
I20260811 00:18:55.896081 18469 heartbeater.cc:511] Master 127.16.71.190:39641 requested a full tablet report, sending...
I20260811 00:18:55.898033 17662 ts_manager.cc:194] Registered new tserver with Master: 8e790a75374d482d972aa097097ce50f (127.16.71.131:36389)
I20260811 00:18:55.898927 17662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:53937
I20260811 00:18:55.904266 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:18:55.925565 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.911220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.16.71.190:39641)
0811 00:18:55.911448 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:55.911456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:55.911887 (+ 431us) server_negotiation.cc:407] Connection header received
0811 00:18:55.912046 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:55.912050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:55.912123 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:55.912217 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:55.913975 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.914796 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.916437 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.916971 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.917088 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.919741 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:55.919747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:55.919750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:55.919794 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:55.922577 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:55.923387 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:55.923391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:55.923394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:55.923553 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:55.923885 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:55.923888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:55.923891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:55.924245 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:55.924445 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:55.925001 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 00:18:55.925209 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:18:55.933372 17662 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:18:55.941737 17662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:18:55.983914 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.957017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:40379 (local address 127.0.0.1:45420)
0811 00:18:55.969353 (+ 12336us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:55.969422 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:55.969731 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:55.972121 (+ 2390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:55.972130 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:55.972170 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:18:55.972701 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.972716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.975826 (+ 3110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.975834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.979866 (+ 4032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.979882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.982436 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.982443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.982710 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.982774 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 00:18:55.982940 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 00:18:55.983205 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12001,"mutex_wait_us":1,"thread_start_us":12951,"threads_started":1}
I20260811 00:18:55.984061 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.957220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45420 (local address 127.16.71.129:40379)
0811 00:18:55.969244 (+ 12024us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:55.969257 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:55.971019 (+ 1762us) server_negotiation.cc:407] Connection header received
0811 00:18:55.971153 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:55.971161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:55.971484 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:55.971727 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:18:55.972937 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.975591 (+ 2654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.980080 (+ 4489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.981818 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.981984 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.982278 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:55.983448 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0811 00:18:55.983654 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11795,"thread_start_us":11628,"threads_started":1}
I20260811 00:18:55.986743 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.957109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:39535 (local address 127.0.0.1:46134)
0811 00:18:55.969675 (+ 12566us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:55.969697 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:55.970022 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:55.978569 (+ 8547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:55.978574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:55.978608 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:18:55.979121 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.979136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.981370 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.981378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.983722 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.983742 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.985844 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.985851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.986070 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.986109 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:18:55.986218 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 00:18:55.986367 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12433,"thread_start_us":12244,"threads_started":1}
I20260811 00:18:55.986855 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.957012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.16.71.130:39535)
0811 00:18:55.977513 (+ 20501us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:55.977524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:55.977561 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:18:55.977679 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:55.977688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:55.978026 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:55.978266 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:18:55.979369 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.980911 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.983995 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.985597 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.985766 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.986018 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:55.986289 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 00:18:55.986477 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20303,"thread_start_us":21623,"threads_started":1}
I20260811 00:18:55.988440 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.970390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36389 (local address 127.0.0.1:39568)
0811 00:18:55.977888 (+ 7498us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:55.977917 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:55.978072 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:55.980088 (+ 2016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:55.980092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:55.980127 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:18:55.980528 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.980542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.982757 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.982761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.985109 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:55.985126 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.987575 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.987582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:55.987772 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.987818 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 00:18:55.987900 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 00:18:55.988053 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7353,"thread_start_us":6914,"threads_started":1}
I20260811 00:18:55.989185 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:55.970712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.16.71.131:36389)
0811 00:18:55.979404 (+ 8692us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:55.979412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:55.979451 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:18:55.979576 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:55.979583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:55.979876 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:55.980094 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:18:55.980740 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.982572 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.985317 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:55.986708 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:55.986873 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:55.987152 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:55.988740 (+ 1588us) server_negotiation.cc:299] Negotiation successful
0811 00:18:55.988911 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8506,"thread_start_us":8318,"threads_started":1}
I20260811 00:18:55.989501 17899 tablet_service.cc:1467] Processing CreateTablet for tablet 2323b5db34b340d48a20c82f847840e5 (DEFAULT_TABLE table=test-table [id=395cd06cbac140bba2b2fd3e62e018b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:18:55.991336 17899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2323b5db34b340d48a20c82f847840e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:55.991227 18115 tablet_service.cc:1467] Processing CreateTablet for tablet 2323b5db34b340d48a20c82f847840e5 (DEFAULT_TABLE table=test-table [id=395cd06cbac140bba2b2fd3e62e018b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:18:55.992964 18115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2323b5db34b340d48a20c82f847840e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:55.997010 18355 tablet_service.cc:1467] Processing CreateTablet for tablet 2323b5db34b340d48a20c82f847840e5 (DEFAULT_TABLE table=test-table [id=395cd06cbac140bba2b2fd3e62e018b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:18:55.998587 18355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2323b5db34b340d48a20c82f847840e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:56.008056 18500 tablet_bootstrap.cc:492] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a: Bootstrap starting.
I20260811 00:18:56.010929 18501 tablet_bootstrap.cc:492] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e: Bootstrap starting.
I20260811 00:18:56.013096 18502 tablet_bootstrap.cc:492] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f: Bootstrap starting.
I20260811 00:18:56.013239 18500 tablet_bootstrap.cc:654] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:56.014961 18500 log.cc:824] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:56.016297 18501 tablet_bootstrap.cc:654] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:56.017406 18502 tablet_bootstrap.cc:654] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:56.017879 18501 log.cc:824] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:56.018273 18500 tablet_bootstrap.cc:492] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a: No bootstrap required, opened a new log
I20260811 00:18:56.018682 18500 ts_tablet_manager.cc:1397] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 00:18:56.018795 18502 log.cc:824] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:56.022034 18501 tablet_bootstrap.cc:492] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e: No bootstrap required, opened a new log
I20260811 00:18:56.022348 18502 tablet_bootstrap.cc:492] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f: No bootstrap required, opened a new log
I20260811 00:18:56.022384 18501 ts_tablet_manager.cc:1397] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 00:18:56.022635 18502 ts_tablet_manager.cc:1397] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 00:18:56.030503 18501 raft_consensus.cc:348] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.031013 18501 raft_consensus.cc:374] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:56.030565 18500 raft_consensus.cc:348] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.031198 18501 raft_consensus.cc:729] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ad9d47853c444c9d02c07bc4273c5e, State: Initialized, Role: FOLLOWER
I20260811 00:18:56.031291 18500 raft_consensus.cc:374] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:56.031405 18500 raft_consensus.cc:729] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96ee2ca851949c080e291a192bd2c5a, State: Initialized, Role: FOLLOWER
I20260811 00:18:56.032160 18501 consensus_queue.cc:260] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.032328 18500 consensus_queue.cc:260] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.033747 18235 heartbeater.cc:503] Master 127.16.71.190:39641 was elected leader, sending a full tablet report...
I20260811 00:18:56.034358 18501 ts_tablet_manager.cc:1428] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e: Time spent starting tablet: real 0.012s user 0.009s sys 0.001s
I20260811 00:18:56.036441 18502 raft_consensus.cc:348] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.037163 18502 raft_consensus.cc:374] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:56.037315 18502 raft_consensus.cc:729] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e790a75374d482d972aa097097ce50f, State: Initialized, Role: FOLLOWER
I20260811 00:18:56.038340 18502 consensus_queue.cc:260] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.041270 18500 ts_tablet_manager.cc:1428] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a: Time spent starting tablet: real 0.022s user 0.012s sys 0.011s
I20260811 00:18:56.042490 18469 heartbeater.cc:503] Master 127.16.71.190:39641 was elected leader, sending a full tablet report...
I20260811 00:18:56.043560 18502 ts_tablet_manager.cc:1428] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f: Time spent starting tablet: real 0.021s user 0.012s sys 0.006s
W20260811 00:18:56.072772 18238 tablet.cc:2369] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:18:56.118881 18508 raft_consensus.cc:482] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:18:56.119318 18508 raft_consensus.cc:504] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
W20260811 00:18:56.121088 18470 tablet.cc:2369] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:18:56.121320 18508 leader_election.cc:288] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27ad9d47853c444c9d02c07bc4273c5e (127.16.71.130:39535), 8e790a75374d482d972aa097097ce50f (127.16.71.131:36389)
I20260811 00:18:56.129667 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.121873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:35933 (local address 127.16.71.130:39535)
0811 00:18:56.122113 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.122118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.122872 (+ 754us) server_negotiation.cc:407] Connection header received
0811 00:18:56.123226 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.123232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.123319 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.123420 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:18:56.124095 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.125091 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.127531 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.128711 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.128866 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.128949 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.129308 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.129453 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:18:56.130005 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.123167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37433 (local address 127.16.71.131:36389)
0811 00:18:56.123334 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.123339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.123792 (+ 453us) server_negotiation.cc:407] Connection header received
0811 00:18:56.123924 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.123929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.124019 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.124105 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:18:56.124846 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.125923 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.128043 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.129089 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.129244 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.129318 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.129627 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.129758 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 00:18:56.130244 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.122780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36389 (local address 127.16.71.129:37433)
0811 00:18:56.123674 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:56.123689 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:56.123801 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:56.124140 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:56.124144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:56.124172 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:18:56.124592 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:56.124605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.126198 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.126202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:56.127842 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:56.127860 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.129327 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.129334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:56.129467 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.129498 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:18:56.129580 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 00:18:56.129720 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":1,"thread_start_us":552,"threads_started":1}
I20260811 00:18:56.130501 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.121881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:39535 (local address 127.16.71.129:35933)
0811 00:18:56.122592 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:56.122611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:56.123010 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:56.123487 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:56.123492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:56.123529 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:18:56.123918 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:56.123932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.125247 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.125253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:56.127289 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:56.127305 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.128927 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.128934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:56.129093 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.129128 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:18:56.129942 (+ 814us) client_negotiation.cc:240] Negotiation successful
0811 00:18:56.130094 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":375,"threads_started":1}
I20260811 00:18:56.131405 18384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323b5db34b340d48a20c82f847840e5" candidate_uuid: "c96ee2ca851949c080e291a192bd2c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e790a75374d482d972aa097097ce50f" is_pre_election: true
I20260811 00:18:56.131407 18150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323b5db34b340d48a20c82f847840e5" candidate_uuid: "c96ee2ca851949c080e291a192bd2c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27ad9d47853c444c9d02c07bc4273c5e" is_pre_election: true
I20260811 00:18:56.131929 18150 raft_consensus.cc:2454] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c96ee2ca851949c080e291a192bd2c5a in term 0.
I20260811 00:18:56.131930 18384 raft_consensus.cc:2454] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c96ee2ca851949c080e291a192bd2c5a in term 0.
I20260811 00:18:56.132691 17822 leader_election.cc:302] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e790a75374d482d972aa097097ce50f, c96ee2ca851949c080e291a192bd2c5a; no voters:
I20260811 00:18:56.133116 18508 raft_consensus.cc:2781] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:18:56.133255 18508 raft_consensus.cc:482] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:18:56.133383 18508 raft_consensus.cc:3037] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:56.135834 18508 raft_consensus.cc:504] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.136771 18508 leader_election.cc:288] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [CANDIDATE]: Term 1 election: Requested vote from peers 27ad9d47853c444c9d02c07bc4273c5e (127.16.71.130:39535), 8e790a75374d482d972aa097097ce50f (127.16.71.131:36389)
I20260811 00:18:56.137411 18150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323b5db34b340d48a20c82f847840e5" candidate_uuid: "c96ee2ca851949c080e291a192bd2c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27ad9d47853c444c9d02c07bc4273c5e"
I20260811 00:18:56.137466 18384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323b5db34b340d48a20c82f847840e5" candidate_uuid: "c96ee2ca851949c080e291a192bd2c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e790a75374d482d972aa097097ce50f"
I20260811 00:18:56.137620 18150 raft_consensus.cc:3037] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:56.137723 18384 raft_consensus.cc:3037] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:56.140107 18150 raft_consensus.cc:2454] T 2323b5db34b340d48a20c82f847840e5 P 27ad9d47853c444c9d02c07bc4273c5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c96ee2ca851949c080e291a192bd2c5a in term 1.
I20260811 00:18:56.140089 18384 raft_consensus.cc:2454] T 2323b5db34b340d48a20c82f847840e5 P 8e790a75374d482d972aa097097ce50f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c96ee2ca851949c080e291a192bd2c5a in term 1.
I20260811 00:18:56.140599 17822 leader_election.cc:302] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27ad9d47853c444c9d02c07bc4273c5e, c96ee2ca851949c080e291a192bd2c5a; no voters:
I20260811 00:18:56.141026 18508 raft_consensus.cc:2781] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:56.141495 18508 raft_consensus.cc:686] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [term 1 LEADER]: Becoming Leader. State: Replica: c96ee2ca851949c080e291a192bd2c5a, State: Running, Role: LEADER
I20260811 00:18:56.142076 18508 consensus_queue.cc:237] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } }
I20260811 00:18:56.147222 17662 catalog_manager.cc:5495] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a reported cstate change: term changed from 0 to 1, leader changed from <none> to c96ee2ca851949c080e291a192bd2c5a (127.16.71.129). New cstate: current_term: 1 leader_uuid: "c96ee2ca851949c080e291a192bd2c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27ad9d47853c444c9d02c07bc4273c5e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c96ee2ca851949c080e291a192bd2c5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e790a75374d482d972aa097097ce50f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36389 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:18:56.185626 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.169075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.16.71.129:40379)
0811 00:18:56.169276 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.169284 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.169860 (+ 576us) server_negotiation.cc:407] Connection header received
0811 00:18:56.170049 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.170053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.170149 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.170240 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:56.171882 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.172743 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.174072 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.174508 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.174627 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.179095 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:56.179114 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:56.179133 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:56.179178 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:56.182339 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.183404 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.183415 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.183417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.183645 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.184049 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.184052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.184054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.184465 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:56.184766 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.185044 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.185289 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[17577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:18:56.250200 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.233725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.16.71.129:40379)
0811 00:18:56.233926 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.233934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.234321 (+ 387us) server_negotiation.cc:407] Connection header received
0811 00:18:56.234508 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.234512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.234584 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.234675 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:56.236212 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.237035 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.238389 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.238832 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.238957 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.243198 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:56.243204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:56.243207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:56.243251 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:56.245901 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.246741 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.246747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.246750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.246993 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.247520 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.247524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.247527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.247952 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:56.248284 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.249652 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.249896 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 00:18:56.273913 18008 tablet.cc:2369] T 2323b5db34b340d48a20c82f847840e5 P c96ee2ca851949c080e291a192bd2c5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[17577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:18:56.320068 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.303042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.16.71.129:40379)
0811 00:18:56.303291 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.303299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.303637 (+ 338us) server_negotiation.cc:407] Connection header received
0811 00:18:56.303817 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.303821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.303912 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.304038 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:56.305641 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.306541 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.308051 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.308548 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.308732 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.313698 (+ 4966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:56.313705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:56.313709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:56.313760 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:56.316667 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.317584 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.317590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.317593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.317777 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.318368 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.318375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.318378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.318842 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:56.319131 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.319503 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.319724 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 00:18:56.322263 17865 tablet_service.cc:2908] Rejecting scan request for tablet 2323b5db34b340d48a20c82f847840e5: Uninitialized: clean time has not yet been initialized
I20260811 00:18:56.326299 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17783
I20260811 00:18:56.342258 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.327375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.16.71.129:40379)
0811 00:18:56.327616 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.327624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.327941 (+ 317us) server_negotiation.cc:407] Connection header received
0811 00:18:56.328102 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.328107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.328180 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.328280 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:56.330008 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.330782 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.332786 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.333347 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.333477 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.335920 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:56.335927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:56.335930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:56.335975 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:56.339190 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.340002 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.340007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.340009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.340178 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.340761 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.340766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.340768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.341130 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:56.341346 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.341659 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.341901 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:18:56.343052 17998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:56.437045 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17783
I20260811 00:18:56.465842 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18016
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:18:56.483615 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.466885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.16.71.130:39535)
0811 00:18:56.467116 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.467124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.467144 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:18:56.467211 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.467215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.467278 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.467366 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:56.468680 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.469455 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.470879 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.471305 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.471429 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.476863 (+ 5434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:56.476883 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:56.476901 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:56.476958 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:56.480066 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.481058 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.481068 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.481070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.481246 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.481778 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.481783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.481785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.482304 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:56.482606 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.483082 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.483301 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 00:18:56.484632 18228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:56.575835 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18016
I20260811 00:18:56.607657 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18251
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[17577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:18:56.623831 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:56.608594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.16.71.131:36389)
0811 00:18:56.608864 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:56.608869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:56.608890 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:18:56.608950 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:56.608954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:56.609029 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:56.609120 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:56.610466 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.611229 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.612579 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:56.613050 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:56.613173 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:56.617907 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:56.617926 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:56.617945 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:56.617996 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:56.620863 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.621647 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.621657 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.621659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.621817 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:56.622227 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:56.622232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:56.622234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:56.622658 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:56.622925 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:56.623229 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 00:18:56.623429 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260811 00:18:56.624655 18458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:56.714531 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18251
I20260811 00:18:56.743299 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17608
I20260811 00:18:56.744562 17721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:56.855484 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17608
2026-08-11T00:18:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4751 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[18542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[18542](info): set up 2 sockets
Aug 11 00:18:56 dist-test-slave-f7th krb5kdc[18542](info): commencing operation
krb5kdc: starting...
W20260811 00:18:58.138662 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:18:58 dist-test-slave-f7th krb5kdc[18542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:18:58.361155 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39707
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:39707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:58.714603 18565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:58.714946 18565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:58.715054 18565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:58.722810 18565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:58.722949 18565 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:18:58.722994 18565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:58.723080 18565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:58.723148 18565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:58.723240 18565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:58.733949 18565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:39707
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:58.735370 18565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:58.736860 18565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:58.749389 18579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:58.749468 18584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:58.749773 18578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:58.750553 18565 server_base.cc:1034] running on GCE node
I20260811 00:18:58.751485 18565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:58.753160 18565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:58.754419 18565 hybrid_clock.cc:648] HybridClock initialized: now 1786407538754343 us; error 46 us; skew 500 ppm
I20260811 00:18:58.754814 18565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:58 dist-test-slave-f7th krb5kdc[18542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407538, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:58.758451 18565 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:18:58.760224 18565 webserver.cc:460] Webserver started at http://127.16.71.190:39905/ using document root <none> and password file <none>
I20260811 00:18:58.761104 18565 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:58.761217 18565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:58.761557 18565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:58.765007 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "a90b7c9d6da544a58433306fd8fe4174"
format_stamp: "Formatted at 2026-08-11 00:18:58 on dist-test-slave-f7th"
server_key: "8e510595b4537623064639b8434737cb"
server_key_iv: "a7e343248bd87f4384e17d02e6cf629e"
server_key_version: "encryptionkey@0"
I20260811 00:18:58.765877 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "a90b7c9d6da544a58433306fd8fe4174"
format_stamp: "Formatted at 2026-08-11 00:18:58 on dist-test-slave-f7th"
server_key: "8e510595b4537623064639b8434737cb"
server_key_iv: "a7e343248bd87f4384e17d02e6cf629e"
server_key_version: "encryptionkey@0"
I20260811 00:18:58.772178 18565 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:18:58.776201 18593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:58.777774 18565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:18:58.777968 18565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "a90b7c9d6da544a58433306fd8fe4174"
format_stamp: "Formatted at 2026-08-11 00:18:58 on dist-test-slave-f7th"
server_key: "8e510595b4537623064639b8434737cb"
server_key_iv: "a7e343248bd87f4384e17d02e6cf629e"
server_key_version: "encryptionkey@0"
I20260811 00:18:58.778223 18565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:58.797744 18565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:58.802227 18565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:58.802475 18565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:58.825472 18565 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:39707
I20260811 00:18:58.825505 18696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:39707 every 8 connection(s)
I20260811 00:18:58.827020 18565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:58.831286 18697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:58.833860 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18565
I20260811 00:18:58.834112 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:18:58.834926 16670 external_mini_cluster.cc:1428] Setting key a47b2fbf9e795c092c6c1392696d1de1
I20260811 00:18:58.845024 18697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174: Bootstrap starting.
I20260811 00:18:58.849442 18697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:58.851011 18697 log.cc:824] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:18:58 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:58.854811 18697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174: No bootstrap required, opened a new log
I20260811 00:18:58.861106 18711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:58.837275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.16.71.190:39707)
0811 00:18:58.838207 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:58.838221 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:58.838271 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:18:58.840164 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:58.840195 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:58.840719 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:58.841128 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:58.843063 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:58.844488 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:58.846334 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:58.847204 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:58.847370 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:58.852926 (+ 5556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:58.852936 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:58.852955 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:58.853006 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:58.856379 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:58.857259 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:58.857267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:58.857278 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:58.857486 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:58.857907 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:58.857911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:58.857914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:58.858538 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:58.858835 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:58.859374 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 00:18:58.859839 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":421,"threads_started":1}
I20260811 00:18:58.863201 18697 raft_consensus.cc:348] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90b7c9d6da544a58433306fd8fe4174" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39707 } }
I20260811 00:18:58.863636 18697 raft_consensus.cc:374] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:58.863745 18697 raft_consensus.cc:729] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90b7c9d6da544a58433306fd8fe4174, State: Initialized, Role: FOLLOWER
I20260811 00:18:58.864421 18697 consensus_queue.cc:260] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90b7c9d6da544a58433306fd8fe4174" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39707 } }
I20260811 00:18:58.864758 18697 raft_consensus.cc:388] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:58.864903 18697 raft_consensus.cc:482] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:58.865072 18697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:58.867440 18697 raft_consensus.cc:504] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90b7c9d6da544a58433306fd8fe4174" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39707 } }
I20260811 00:18:58.868074 18697 leader_election.cc:302] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a90b7c9d6da544a58433306fd8fe4174; no voters:
I20260811 00:18:58.868834 18697 leader_election.cc:288] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:58.868949 18713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:58.870198 18713 raft_consensus.cc:686] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [term 1 LEADER]: Becoming Leader. State: Replica: a90b7c9d6da544a58433306fd8fe4174, State: Running, Role: LEADER
I20260811 00:18:58.870766 18713 consensus_queue.cc:237] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90b7c9d6da544a58433306fd8fe4174" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39707 } }
I20260811 00:18:58.871500 18697 sys_catalog.cc:564] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:58.873947 18718 sys_catalog.cc:455] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a90b7c9d6da544a58433306fd8fe4174. Latest consensus state: current_term: 1 leader_uuid: "a90b7c9d6da544a58433306fd8fe4174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90b7c9d6da544a58433306fd8fe4174" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39707 } } }
I20260811 00:18:58.874405 18718 sys_catalog.cc:458] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:58.875099 18714 sys_catalog.cc:455] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a90b7c9d6da544a58433306fd8fe4174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90b7c9d6da544a58433306fd8fe4174" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39707 } } }
I20260811 00:18:58.875423 18714 sys_catalog.cc:458] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:58.878048 18729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:58.885049 18729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:58.895632 18729 catalog_manager.cc:1347] Generated new cluster ID: 6572b78681e147169b89a45ce11eda4e
I20260811 00:18:58.895773 18729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:58.917349 18729 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:58.918843 18729 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:58.926571 18729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174: Generated new TSK 0
I20260811 00:18:58.927802 18729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:18:58.997097 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39707
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:59.362233 18758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:59.362572 18758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:59.362673 18758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:59.370333 18758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:59.370473 18758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:59.370559 18758 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:18:59.370641 18758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:18:59.381263 18758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:39707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:59.382709 18758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:59.384123 18758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:59.398841 18765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:59.398880 18764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:59.398965 18767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:59.400084 18758 server_base.cc:1034] running on GCE node
I20260811 00:18:59.400902 18758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:59.402030 18758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:59.403317 18758 hybrid_clock.cc:648] HybridClock initialized: now 1786407539403230 us; error 47 us; skew 500 ppm
I20260811 00:18:59.403707 18758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:59 dist-test-slave-f7th krb5kdc[18542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407539, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:59.412127 18758 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:18:59.414083 18758 webserver.cc:460] Webserver started at http://127.16.71.129:41051/ using document root <none> and password file <none>
I20260811 00:18:59.414866 18758 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:59.414975 18758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:59.415290 18758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:59.418653 18758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "796d5b9265164550a63e3dfd509478f0"
format_stamp: "Formatted at 2026-08-11 00:18:59 on dist-test-slave-f7th"
server_key: "f7c609d63a65e9d92f7700e8fb721804"
server_key_iv: "d4bf717228814750c9ffb6a47c12d3f5"
server_key_version: "encryptionkey@0"
I20260811 00:18:59.419685 18758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "796d5b9265164550a63e3dfd509478f0"
format_stamp: "Formatted at 2026-08-11 00:18:59 on dist-test-slave-f7th"
server_key: "f7c609d63a65e9d92f7700e8fb721804"
server_key_iv: "d4bf717228814750c9ffb6a47c12d3f5"
server_key_version: "encryptionkey@0"
I20260811 00:18:59.425711 18758 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:18:59.429631 18780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:59.431197 18758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260811 00:18:59.431389 18758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "796d5b9265164550a63e3dfd509478f0"
format_stamp: "Formatted at 2026-08-11 00:18:59 on dist-test-slave-f7th"
server_key: "f7c609d63a65e9d92f7700e8fb721804"
server_key_iv: "d4bf717228814750c9ffb6a47c12d3f5"
server_key_version: "encryptionkey@0"
I20260811 00:18:59.431622 18758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:59.447140 18758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:59.452128 18758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:59.452431 18758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:59.455536 18758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:18:59.455643 18758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:59.455787 18758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:18:59.455850 18758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:18:59 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407539, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:18:59.483302 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:59.456295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:44925 (local address 127.16.71.190:39707)
0811 00:18:59.457108 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:59.457115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:59.458262 (+ 1147us) server_negotiation.cc:407] Connection header received
0811 00:18:59.461422 (+ 3160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:59.461429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:59.461537 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:59.461661 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:59.464297 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.465497 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.468048 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.468646 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.468788 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:59.474427 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:59.474436 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:59.474441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:59.474502 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:59.478140 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:59.479693 (+ 1553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:59.479700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:59.479703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:59.479897 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:59.480786 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:59.480792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:59.480795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:59.481257 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:59.481407 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:59.482644 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0811 00:18:59.482922 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":539,"threads_started":1}
I20260811 00:18:59.484517 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:59.457128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39707 (local address 127.16.71.129:44925)
0811 00:18:59.458036 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:59.458100 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:59.460922 (+ 2822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:59.462067 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:59.462081 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:59.463000 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:59.464032 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:59.464060 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.465716 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.465721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:59.467758 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:59.467782 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.468936 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.468945 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:59.469196 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:59.470086 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:59.470130 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:59.474044 (+ 3914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:59.478376 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:59.478393 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:59.478424 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:59.479435 (+ 1011us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:59.480113 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:59.480118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:59.480122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:59.480568 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:59.481461 (+ 893us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:59.481471 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:59.482013 (+ 542us) client_negotiation.cc:769] Sending connection context
0811 00:18:59.482508 (+ 495us) client_negotiation.cc:240] Negotiation successful
0811 00:18:59.483156 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":337,"threads_started":1}
I20260811 00:18:59.507535 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:59.491501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:56417 (local address 127.16.71.190:39707)
0811 00:18:59.491706 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:59.491712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:59.492500 (+ 788us) server_negotiation.cc:407] Connection header received
0811 00:18:59.492811 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:59.492816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:59.492900 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:59.493015 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:59.494832 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.495590 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.497738 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.498208 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.498340 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:59.500293 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:59.500298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:59.500301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:59.500335 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:59.503776 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:59.504957 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:59.504964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:59.504967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:59.505168 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:59.505612 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:59.505618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:59.505621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:59.506086 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:59.506268 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:59.507006 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 00:18:59.507225 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:18:59.507877 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:59.491515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39707 (local address 127.16.71.129:56417)
0811 00:18:59.492327 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:59.492370 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:59.492571 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:59.493077 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:59.493083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:59.494033 (+ 950us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:59.494609 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:59.494627 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.495785 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.495789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:59.497543 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:59.497566 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:59.498430 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:59.498439 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:59.498604 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:59.499260 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:59.499279 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:59.500063 (+ 784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:59.503968 (+ 3905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:59.503974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:59.503979 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:59.504765 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:59.505315 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:59.505319 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:59.505321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:59.505470 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:59.506345 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:59.506351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:59.506627 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 00:18:59.506858 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 00:18:59.507328 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":430,"threads_started":1}
I20260811 00:18:59.508291 18758 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:33133
I20260811 00:18:59.508318 18999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:33133 every 8 connection(s)
I20260811 00:18:59.509910 18758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:18:59.512245 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18758
I20260811 00:18:59.512487 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:18:59.512472 19002 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39707
I20260811 00:18:59.512831 19002 heartbeater.cc:464] Registering TS with master...
I20260811 00:18:59.513412 16670 external_mini_cluster.cc:1428] Setting key ddec23fc104fc3f3055d2ac2d158322e
I20260811 00:18:59.513598 19002 heartbeater.cc:511] Master 127.16.71.190:39707 requested a full tablet report, sending...
I20260811 00:18:59.515848 18620 ts_manager.cc:194] Registered new tserver with Master: 796d5b9265164550a63e3dfd509478f0 (127.16.71.129:33133)
I20260811 00:18:59.517513 18620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:44925
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:18:59.560600 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39707
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:59.912807 19006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:59.913167 19006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:59.913314 19006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:59.920858 19006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:59.921011 19006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:59.921141 19006 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:18:59.921254 19006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:18:59.931799 19006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:39707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:59.933305 19006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:59.934836 19006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:59.950945 19022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:59.950886 19006 server_base.cc:1034] running on GCE node
W20260811 00:18:59.950910 19016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:59.950886 19017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:59.952159 19006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:59.953311 19006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:59.954581 19006 hybrid_clock.cc:648] HybridClock initialized: now 1786407539954484 us; error 71 us; skew 500 ppm
I20260811 00:18:59.954923 19006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:59 dist-test-slave-f7th krb5kdc[18542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407539, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:59.958415 19006 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:18:59.960139 19006 webserver.cc:460] Webserver started at http://127.16.71.130:37119/ using document root <none> and password file <none>
I20260811 00:18:59.961018 19006 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:59.961254 19006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:59.961644 19006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:59.964846 19006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "b226d34f5c0049c9b03002a4fef0679c"
format_stamp: "Formatted at 2026-08-11 00:18:59 on dist-test-slave-f7th"
server_key: "2bad3478410a209aa24f9f237db142e9"
server_key_iv: "aafe382f8ce2e4827073b68f544e816e"
server_key_version: "encryptionkey@0"
I20260811 00:18:59.965682 19006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "b226d34f5c0049c9b03002a4fef0679c"
format_stamp: "Formatted at 2026-08-11 00:18:59 on dist-test-slave-f7th"
server_key: "2bad3478410a209aa24f9f237db142e9"
server_key_iv: "aafe382f8ce2e4827073b68f544e816e"
server_key_version: "encryptionkey@0"
I20260811 00:18:59.971561 19006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 00:18:59.975500 19035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:59.977069 19006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:18:59.977247 19006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "b226d34f5c0049c9b03002a4fef0679c"
format_stamp: "Formatted at 2026-08-11 00:18:59 on dist-test-slave-f7th"
server_key: "2bad3478410a209aa24f9f237db142e9"
server_key_iv: "aafe382f8ce2e4827073b68f544e816e"
server_key_version: "encryptionkey@0"
I20260811 00:18:59.977480 19006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:59.996744 19006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:00.001266 19006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:00.001559 19006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:00.004535 19006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:00.004658 19006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:00.004812 19006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:00.004873 19006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:00 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407539, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:00.030402 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.005253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:32959 (local address 127.16.71.190:39707)
0811 00:19:00.005493 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.005501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.006984 (+ 1483us) server_negotiation.cc:407] Connection header received
0811 00:19:00.009367 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.009373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.009475 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.009603 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:00.012149 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.013452 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.015252 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.015933 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.016082 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.021939 (+ 5857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:00.021948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:00.021953 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:00.022002 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:00.025924 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.027099 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.027105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.027109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.027275 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.027976 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.027983 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.027986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.028368 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:00.028548 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.029744 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.030012 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:19:00.031605 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.005809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39707 (local address 127.16.71.130:32959)
0811 00:19:00.006739 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.006797 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.008999 (+ 2202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.010059 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.010068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.010950 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:00.011897 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.011930 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.013635 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.013640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.015092 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.015106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.016431 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.016449 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.017013 (+ 564us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.017957 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:00.017999 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:00.021544 (+ 3545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:00.026137 (+ 4593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.026154 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.026180 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.026933 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.027447 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.027456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.027460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.027786 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.028686 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:00.028699 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:00.029221 (+ 522us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.029635 (+ 414us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.030292 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":372,"threads_started":1}
I20260811 00:19:00.055410 19006 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:38721
I20260811 00:19:00.055428 19245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:38721 every 8 connection(s)
I20260811 00:19:00.055868 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.039025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:54833 (local address 127.16.71.190:39707)
0811 00:19:00.039244 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.039251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.039892 (+ 641us) server_negotiation.cc:407] Connection header received
0811 00:19:00.040088 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.040094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.040194 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.040323 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:00.041565 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.042732 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.045425 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.045905 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.046031 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.048730 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:00.048737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:00.048740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:00.048780 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:00.052179 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.053227 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.053234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.053237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.053433 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.053840 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.053845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.053848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.054362 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:00.054532 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.055260 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.055556 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:19:00.055997 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.039082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39707 (local address 127.16.71.130:54833)
0811 00:19:00.039725 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.039745 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.039947 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.040430 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.040434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.041034 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:00.041376 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.041388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.043197 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.043203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.045172 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.045200 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.046132 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.046139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.046349 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.047101 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:00.047128 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:00.048367 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:00.052362 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.052368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.052372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.053048 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.053560 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.053564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.053566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.053703 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.054600 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:00.054606 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:00.054918 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.055092 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.055544 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":333,"threads_started":1}
I20260811 00:19:00.057168 19006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:00.060176 19246 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39707
I20260811 00:19:00.060436 19246 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:00.061213 19246 heartbeater.cc:511] Master 127.16.71.190:39707 requested a full tablet report, sending...
I20260811 00:19:00.062721 18620 ts_manager.cc:194] Registered new tserver with Master: b226d34f5c0049c9b03002a4fef0679c (127.16.71.130:38721)
I20260811 00:19:00.063580 18620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:32959
I20260811 00:19:00.064544 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19006
I20260811 00:19:00.064774 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:00.065199 16670 external_mini_cluster.cc:1428] Setting key 01871e526b200ab08865b509579b68c3
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:19:00.117291 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39707
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:00.479257 19254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:00.479601 19254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:00.479707 19254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:00.487612 19254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:00.487751 19254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:00.487839 19254 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:19:00.487927 19254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:00.498387 19254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:39707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:00.499853 19254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:00.501286 19254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:00.516283 19263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:00.516505 19268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:00.516306 19264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:00.517129 19254 server_base.cc:1034] running on GCE node
I20260811 00:19:00.517846 19254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:00.518919 19254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:00.520157 19254 hybrid_clock.cc:648] HybridClock initialized: now 1786407540520077 us; error 52 us; skew 500 ppm
I20260811 00:19:00.520494 19254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:00.520931 19002 heartbeater.cc:503] Master 127.16.71.190:39707 was elected leader, sending a full tablet report...
Aug 11 00:19:00 dist-test-slave-f7th krb5kdc[18542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407540, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:00.524562 19254 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:19:00.526342 19254 webserver.cc:460] Webserver started at http://127.16.71.131:38421/ using document root <none> and password file <none>
I20260811 00:19:00.527128 19254 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:00.527231 19254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:00.527516 19254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:00.530802 19254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "ec46bde515fc4de883748f69859ec5f5"
format_stamp: "Formatted at 2026-08-11 00:19:00 on dist-test-slave-f7th"
server_key: "c57eaed0f743b9a2257c44f643c84094"
server_key_iv: "40c12b35740c754042c87bccfb09ebe2"
server_key_version: "encryptionkey@0"
I20260811 00:19:00.531805 19254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "ec46bde515fc4de883748f69859ec5f5"
format_stamp: "Formatted at 2026-08-11 00:19:00 on dist-test-slave-f7th"
server_key: "c57eaed0f743b9a2257c44f643c84094"
server_key_iv: "40c12b35740c754042c87bccfb09ebe2"
server_key_version: "encryptionkey@0"
I20260811 00:19:00.538309 19254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260811 00:19:00.542753 19279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:00.544737 19254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:19:00.544968 19254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "ec46bde515fc4de883748f69859ec5f5"
format_stamp: "Formatted at 2026-08-11 00:19:00 on dist-test-slave-f7th"
server_key: "c57eaed0f743b9a2257c44f643c84094"
server_key_iv: "40c12b35740c754042c87bccfb09ebe2"
server_key_version: "encryptionkey@0"
I20260811 00:19:00.545220 19254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:00.574153 19254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:00.578912 19254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:00.579198 19254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:00.582409 19254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:00.582531 19254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:00.582660 19254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:00.582736 19254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:00 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407540, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:00.609577 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.583224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:54411 (local address 127.16.71.190:39707)
0811 00:19:00.583768 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.583773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.585306 (+ 1533us) server_negotiation.cc:407] Connection header received
0811 00:19:00.587686 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.587695 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.587809 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.587941 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:00.590664 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.591876 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.595442 (+ 3566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.595958 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.596121 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.601449 (+ 5328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:00.601455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:00.601459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:00.601509 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:00.604835 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.606108 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.606113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.606116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.606276 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.606975 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.606981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.606984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.607439 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:00.607613 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.608868 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.609147 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":325,"threads_started":1}
I20260811 00:19:00.610864 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.584061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39707 (local address 127.16.71.131:54411)
0811 00:19:00.585113 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.585161 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.587271 (+ 2110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.588373 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.588388 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.589341 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:00.590432 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.590458 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.592107 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.592118 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.594942 (+ 2824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.594972 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.596296 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.596305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.596698 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.597705 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:00.597743 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:00.601099 (+ 3356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:00.605022 (+ 3923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.605035 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.605059 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.605874 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.606443 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.606452 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.606456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.606766 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.607713 (+ 947us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:00.607723 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:00.608328 (+ 605us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.608742 (+ 414us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.609414 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":464,"threads_started":1}
I20260811 00:19:00.630064 19254 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:43831
I20260811 00:19:00.630081 19493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:43831 every 8 connection(s)
I20260811 00:19:00.631731 19254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:00.634251 19494 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39707
I20260811 00:19:00.634626 19494 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:00.635552 19494 heartbeater.cc:511] Master 127.16.71.190:39707 requested a full tablet report, sending...
I20260811 00:19:00.637517 18620 ts_manager.cc:194] Registered new tserver with Master: ec46bde515fc4de883748f69859ec5f5 (127.16.71.131:43831)
I20260811 00:19:00.637677 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.620173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39707 (local address 127.16.71.131:38515)
0811 00:19:00.621149 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.621171 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.621324 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.621739 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.621745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.622430 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:00.622992 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.623007 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.624361 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.624369 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.626664 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.626696 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.628181 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.628187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.628365 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.629237 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:00.629267 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:00.630540 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:00.634386 (+ 3846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.634393 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.634397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.635102 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.635667 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:00.635673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:00.635677 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:00.635860 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:00.636657 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:00.636662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:00.636903 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.637056 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.637352 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":666,"threads_started":1}
I20260811 00:19:00.637864 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.620211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:38515 (local address 127.16.71.190:39707)
0811 00:19:00.620425 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.620432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.621291 (+ 859us) server_negotiation.cc:407] Connection header received
0811 00:19:00.621435 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.621441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.621552 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.621683 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:00.623255 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.624191 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.626989 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.627735 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.627850 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.630785 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:00.630790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:00.630794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:00.630832 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:00.634214 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.635275 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.635282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.635285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.635494 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.636008 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.636014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.636017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.636403 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:00.636561 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.637209 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.637508 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:19:00.638618 18620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:54411
I20260811 00:19:00.641705 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19254
I20260811 00:19:00.641906 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:00.642336 16670 external_mini_cluster.cc:1428] Setting key ef5484fadd6993880f566edc69e26abe
I20260811 00:19:00.645617 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:19:00 dist-test-slave-f7th krb5kdc[18542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:19:00 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:00.699954 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.684723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.16.71.190:39707)
0811 00:19:00.684931 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.684936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.685247 (+ 311us) server_negotiation.cc:407] Connection header received
0811 00:19:00.685457 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.685461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.685532 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.685616 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:00.687175 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.688034 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.689432 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.689857 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.690031 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.694211 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:00.694217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:00.694221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:00.694265 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:00.697217 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.698005 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.698011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.698014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.698161 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:00.698605 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:00.698608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:00.698610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:00.698902 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:00.699160 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.699451 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.699653 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:19:00.704540 18620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:00.712070 18620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:00.741237 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.726097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:33133 (local address 127.0.0.1:59892)
0811 00:19:00.726985 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.727027 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.727239 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.728450 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.728457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.728495 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:00.728924 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.728939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.733156 (+ 4217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.733164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.735979 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.736010 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.740052 (+ 4042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.740059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.740307 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.740391 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.740559 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.740813 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":432,"threads_started":1}
I20260811 00:19:00.741321 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.726325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.16.71.129:33133)
0811 00:19:00.727556 (+ 1231us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.727565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.727592 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:19:00.727700 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.727706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.727978 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.730678 (+ 2700us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:00.730963 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.732823 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.738105 (+ 5282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.739811 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.740009 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.740255 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.740823 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.741018 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":920,"threads_started":1}
I20260811 00:19:00.745859 18878 tablet_service.cc:1467] Processing CreateTablet for tablet a3301648853a4a8f883f8ee2da7dcb39 (DEFAULT_TABLE table=test-table [id=3452fd1fe2304b2aa30515f621566025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:00.748368 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.728237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43831 (local address 127.0.0.1:57454)
0811 00:19:00.738415 (+ 10178us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.738446 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.738783 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.740740 (+ 1957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.740745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.740777 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:00.741212 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.741229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.743650 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.743654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.745412 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.745429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.747437 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.747445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.747641 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.747678 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.747774 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.747926 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10006,"thread_start_us":10649,"threads_started":1}
I20260811 00:19:00.748363 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.728922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.16.71.131:43831)
0811 00:19:00.739794 (+ 10872us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.739805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.739850 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:19:00.739974 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.739984 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.740322 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.741450 (+ 1128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:00.742165 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.743506 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.745606 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.747199 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.747363 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.747646 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.747839 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.748024 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10649,"thread_start_us":10412,"threads_started":1}
I20260811 00:19:00.749030 19518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.727579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:38721 (local address 127.0.0.1:60740)
0811 00:19:00.728127 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.728147 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.728395 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.742304 (+ 13909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.742310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.742338 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:00.742709 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.742723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.744474 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.744479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.746220 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.746237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.748042 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.748049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.748239 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.748276 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.748373 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.748513 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":273,"threads_started":1}
I20260811 00:19:00.749249 19520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.727857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.16.71.130:38721)
0811 00:19:00.740754 (+ 12897us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.740762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.740789 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:19:00.740884 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.740891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.741882 (+ 991us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.742093 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:00.742914 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.744309 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.746422 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.747552 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.747689 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.747883 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.748807 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.748984 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12708,"thread_start_us":12596,"threads_started":1}
I20260811 00:19:00.749683 18878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3301648853a4a8f883f8ee2da7dcb39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:00.752327 19375 tablet_service.cc:1467] Processing CreateTablet for tablet a3301648853a4a8f883f8ee2da7dcb39 (DEFAULT_TABLE table=test-table [id=3452fd1fe2304b2aa30515f621566025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:00.753114 19132 tablet_service.cc:1467] Processing CreateTablet for tablet a3301648853a4a8f883f8ee2da7dcb39 (DEFAULT_TABLE table=test-table [id=3452fd1fe2304b2aa30515f621566025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:00.753970 19375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3301648853a4a8f883f8ee2da7dcb39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:00.754652 19132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3301648853a4a8f883f8ee2da7dcb39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:00.766182 19529 tablet_bootstrap.cc:492] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Bootstrap starting.
I20260811 00:19:00.768607 19531 tablet_bootstrap.cc:492] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Bootstrap starting.
I20260811 00:19:00.770861 19529 tablet_bootstrap.cc:654] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:00.772221 19532 tablet_bootstrap.cc:492] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Bootstrap starting.
I20260811 00:19:00.772432 19529 log.cc:824] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:00.772945 19531 tablet_bootstrap.cc:654] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:00.777217 19532 tablet_bootstrap.cc:654] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:00.777412 19531 log.cc:824] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:00.778743 19532 log.cc:824] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:00.783639 19532 tablet_bootstrap.cc:492] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: No bootstrap required, opened a new log
I20260811 00:19:00.783663 19531 tablet_bootstrap.cc:492] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: No bootstrap required, opened a new log
I20260811 00:19:00.784020 19532 ts_tablet_manager.cc:1397] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 00:19:00.784200 19531 ts_tablet_manager.cc:1397] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.000s
I20260811 00:19:00.785048 19529 tablet_bootstrap.cc:492] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: No bootstrap required, opened a new log
I20260811 00:19:00.785352 19529 ts_tablet_manager.cc:1397] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.005s
I20260811 00:19:00.792318 19532 raft_consensus.cc:348] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.792853 19532 raft_consensus.cc:374] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:00.792956 19532 raft_consensus.cc:729] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b226d34f5c0049c9b03002a4fef0679c, State: Initialized, Role: FOLLOWER
I20260811 00:19:00.793790 19532 consensus_queue.cc:260] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.796447 19246 heartbeater.cc:503] Master 127.16.71.190:39707 was elected leader, sending a full tablet report...
I20260811 00:19:00.797199 19532 ts_tablet_manager.cc:1428] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 00:19:00.797308 19531 raft_consensus.cc:348] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.797822 19531 raft_consensus.cc:374] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:00.797966 19531 raft_consensus.cc:729] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796d5b9265164550a63e3dfd509478f0, State: Initialized, Role: FOLLOWER
I20260811 00:19:00.798615 19531 consensus_queue.cc:260] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.799423 19529 raft_consensus.cc:348] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.799896 19529 raft_consensus.cc:374] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:00.800012 19529 raft_consensus.cc:729] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec46bde515fc4de883748f69859ec5f5, State: Initialized, Role: FOLLOWER
I20260811 00:19:00.800513 19531 ts_tablet_manager.cc:1428] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260811 00:19:00.800838 19529 consensus_queue.cc:260] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.802140 19494 heartbeater.cc:503] Master 127.16.71.190:39707 was elected leader, sending a full tablet report...
I20260811 00:19:00.803493 19529 ts_tablet_manager.cc:1428] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Time spent starting tablet: real 0.018s user 0.011s sys 0.004s
W20260811 00:19:00.809497 19248 tablet.cc:2369] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:00.815939 19539 raft_consensus.cc:482] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:00.816293 19539 raft_consensus.cc:504] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.818171 19539 leader_election.cc:288] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 796d5b9265164550a63e3dfd509478f0 (127.16.71.129:33133), ec46bde515fc4de883748f69859ec5f5 (127.16.71.131:43831)
I20260811 00:19:00.826122 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.818658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:60479 (local address 127.16.71.129:33133)
0811 00:19:00.818918 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.818923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.819639 (+ 716us) server_negotiation.cc:407] Connection header received
0811 00:19:00.819741 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.819746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.819899 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.820018 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:00.820738 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.821721 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.823850 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.825021 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.825189 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.825270 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.825713 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.825865 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:19:00.826488 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.818621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:33133 (local address 127.16.71.130:60479)
0811 00:19:00.819349 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.819370 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.819530 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.820067 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.820072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.820105 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:00.820507 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.820521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.821875 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.821882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.823643 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.823658 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.825289 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.825296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.825497 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.825537 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.825657 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.825817 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":371,"threads_started":1}
I20260811 00:19:00.826786 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.819738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:37861 (local address 127.16.71.131:43831)
0811 00:19:00.819958 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.819964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.820306 (+ 342us) server_negotiation.cc:407] Connection header received
0811 00:19:00.820503 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.820509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.820592 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.820720 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:00.821430 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.822464 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.824458 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.825580 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.825739 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.825809 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.826394 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.826544 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:19:00.827471 19547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.819549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43831 (local address 127.16.71.130:37861)
0811 00:19:00.820180 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:00.820196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:00.820364 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:00.820741 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:00.820745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:00.820772 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:00.821231 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.821245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.822627 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.822632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.824261 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:00.824278 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.825991 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.825998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:00.826188 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.826225 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:19:00.826998 (+ 773us) client_negotiation.cc:240] Negotiation successful
0811 00:19:00.827136 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"spinlock_wait_cycles":28544,"thread_start_us":286,"threads_started":1}
I20260811 00:19:00.827958 18918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3301648853a4a8f883f8ee2da7dcb39" candidate_uuid: "b226d34f5c0049c9b03002a4fef0679c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "796d5b9265164550a63e3dfd509478f0" is_pre_election: true
I20260811 00:19:00.828593 18918 raft_consensus.cc:2454] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b226d34f5c0049c9b03002a4fef0679c in term 0.
I20260811 00:19:00.828658 19408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3301648853a4a8f883f8ee2da7dcb39" candidate_uuid: "b226d34f5c0049c9b03002a4fef0679c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec46bde515fc4de883748f69859ec5f5" is_pre_election: true
I20260811 00:19:00.829272 19408 raft_consensus.cc:2454] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b226d34f5c0049c9b03002a4fef0679c in term 0.
I20260811 00:19:00.829432 19042 leader_election.cc:302] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 796d5b9265164550a63e3dfd509478f0, b226d34f5c0049c9b03002a4fef0679c; no voters:
I20260811 00:19:00.829885 19539 raft_consensus.cc:2781] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:00.830054 19539 raft_consensus.cc:482] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:00.830160 19539 raft_consensus.cc:3037] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:00.833021 19539 raft_consensus.cc:504] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.833734 19539 leader_election.cc:288] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [CANDIDATE]: Term 1 election: Requested vote from peers 796d5b9265164550a63e3dfd509478f0 (127.16.71.129:33133), ec46bde515fc4de883748f69859ec5f5 (127.16.71.131:43831)
I20260811 00:19:00.834395 18918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3301648853a4a8f883f8ee2da7dcb39" candidate_uuid: "b226d34f5c0049c9b03002a4fef0679c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "796d5b9265164550a63e3dfd509478f0"
I20260811 00:19:00.834661 18918 raft_consensus.cc:3037] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:00.834739 19408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3301648853a4a8f883f8ee2da7dcb39" candidate_uuid: "b226d34f5c0049c9b03002a4fef0679c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec46bde515fc4de883748f69859ec5f5"
I20260811 00:19:00.834983 19408 raft_consensus.cc:3037] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:00.837425 18918 raft_consensus.cc:2454] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b226d34f5c0049c9b03002a4fef0679c in term 1.
I20260811 00:19:00.837950 19042 leader_election.cc:302] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 796d5b9265164550a63e3dfd509478f0, b226d34f5c0049c9b03002a4fef0679c; no voters:
I20260811 00:19:00.838054 19408 raft_consensus.cc:2454] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b226d34f5c0049c9b03002a4fef0679c in term 1.
I20260811 00:19:00.838277 19539 raft_consensus.cc:2781] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:00.838843 19539 raft_consensus.cc:686] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 1 LEADER]: Becoming Leader. State: Replica: b226d34f5c0049c9b03002a4fef0679c, State: Running, Role: LEADER
I20260811 00:19:00.839471 19539 consensus_queue.cc:237] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.845233 18620 catalog_manager.cc:5495] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c reported cstate change: term changed from 0 to 1, leader changed from <none> to b226d34f5c0049c9b03002a4fef0679c (127.16.71.130). New cstate: current_term: 1 leader_uuid: "b226d34f5c0049c9b03002a4fef0679c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:00.877353 19520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:00.871087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.16.71.130:38721)
0811 00:19:00.871333 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:00.871340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:00.871525 (+ 185us) server_negotiation.cc:407] Connection header received
0811 00:19:00.871868 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:00.871872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:00.871987 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:00.872120 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:00.872528 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.873436 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.874905 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:00.875334 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:00.875461 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:00.876057 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:00.876693 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:00.876856 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:00.876984 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 00:19:00.877080 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 00:19:00.885315 19495 tablet.cc:2369] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:00.886876 18918 raft_consensus.cc:1261] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Refusing update from remote peer b226d34f5c0049c9b03002a4fef0679c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:00.886875 19408 raft_consensus.cc:1261] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Refusing update from remote peer b226d34f5c0049c9b03002a4fef0679c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:00.887988 19549 consensus_queue.cc:1035] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:00.888499 19539 consensus_queue.cc:1035] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [LEADER]: Connected to new peer: Peer: permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:00.904543 19553 mvcc.cc:205] Tried to move back new op lower bound from 7317125287460814848 to 7317125287282716672. Current Snapshot: MvccSnapshot[applied={T|T < 7317125287460814848}]
I20260811 00:19:00.908849 19555 mvcc.cc:205] Tried to move back new op lower bound from 7317125287460814848 to 7317125287282716672. Current Snapshot: MvccSnapshot[applied={T|T < 7317125287460814848}]
I20260811 00:19:00.929471 18620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:00.929908 18620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:00.935106 18620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174: Sending DeleteTablet for 3 replicas of tablet a3301648853a4a8f883f8ee2da7dcb39
I20260811 00:19:00.936761 18878 tablet_service.cc:1514] Processing DeleteTablet for tablet a3301648853a4a8f883f8ee2da7dcb39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:00 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:59892
I20260811 00:19:00.937167 19132 tablet_service.cc:1514] Processing DeleteTablet for tablet a3301648853a4a8f883f8ee2da7dcb39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:00 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:60740
I20260811 00:19:00.937219 19375 tablet_service.cc:1514] Processing DeleteTablet for tablet a3301648853a4a8f883f8ee2da7dcb39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:00 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:57454
I20260811 00:19:00.942477 18620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:00.943346 18620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:00.948333 19574 tablet_replica.cc:330] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: stopping tablet replica
I20260811 00:19:00.948341 19575 tablet_replica.cc:330] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: stopping tablet replica
I20260811 00:19:00.949617 19573 tablet_replica.cc:330] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: stopping tablet replica
I20260811 00:19:00.949542 19575 raft_consensus.cc:2229] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:00.950227 19573 raft_consensus.cc:2229] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:00.950307 19575 raft_consensus.cc:2258] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:00.950754 19573 raft_consensus.cc:2258] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:00.951323 19574 raft_consensus.cc:2229] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:00.954276 19574 raft_consensus.cc:2258] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:00.954276 19575 ts_tablet_manager.cc:1905] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:00.956604 19573 ts_tablet_manager.cc:1905] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:00.957677 19375 tablet_service.cc:1467] Processing CreateTablet for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f (DEFAULT_TABLE table=test-table [id=3b46e80a333f49c99df265880415d0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:00.958248 19375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:00.959937 19574 ts_tablet_manager.cc:1905] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:00.961633 19132 tablet_service.cc:1467] Processing CreateTablet for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f (DEFAULT_TABLE table=test-table [id=3b46e80a333f49c99df265880415d0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:00.962215 19132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:00.962558 19575 ts_tablet_manager.cc:1918] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:00.962777 19575 log.cc:1196] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/a3301648853a4a8f883f8ee2da7dcb39
I20260811 00:19:00.963373 19575 ts_tablet_manager.cc:1939] T a3301648853a4a8f883f8ee2da7dcb39 P ec46bde515fc4de883748f69859ec5f5: Deleting consensus metadata
I20260811 00:19:00.965305 19573 ts_tablet_manager.cc:1918] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:00.965623 19573 log.cc:1196] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/a3301648853a4a8f883f8ee2da7dcb39
I20260811 00:19:00.966286 19573 ts_tablet_manager.cc:1939] T a3301648853a4a8f883f8ee2da7dcb39 P 796d5b9265164550a63e3dfd509478f0: Deleting consensus metadata
I20260811 00:19:00.968659 18600 catalog_manager.cc:4841] TS 796d5b9265164550a63e3dfd509478f0 (127.16.71.129:33133): tablet a3301648853a4a8f883f8ee2da7dcb39 (table test-table [id=3452fd1fe2304b2aa30515f621566025]) successfully deleted
I20260811 00:19:00.969435 18878 tablet_service.cc:1467] Processing CreateTablet for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f (DEFAULT_TABLE table=test-table [id=3b46e80a333f49c99df265880415d0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:00.969910 18878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:00.970232 18600 catalog_manager.cc:4841] TS ec46bde515fc4de883748f69859ec5f5 (127.16.71.131:43831): tablet a3301648853a4a8f883f8ee2da7dcb39 (table test-table [id=3452fd1fe2304b2aa30515f621566025]) successfully deleted
I20260811 00:19:00.971709 19529 tablet_bootstrap.cc:492] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: Bootstrap starting.
I20260811 00:19:00.973549 19529 tablet_bootstrap.cc:654] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:00.975399 19532 tablet_bootstrap.cc:492] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: Bootstrap starting.
I20260811 00:19:00.976313 19531 tablet_bootstrap.cc:492] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Bootstrap starting.
I20260811 00:19:00.977173 19532 tablet_bootstrap.cc:654] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:00.978087 19531 tablet_bootstrap.cc:654] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:00.978453 19529 tablet_bootstrap.cc:492] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: No bootstrap required, opened a new log
I20260811 00:19:00.978616 19529 ts_tablet_manager.cc:1397] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 00:19:00.979153 19574 ts_tablet_manager.cc:1918] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:00.979429 19574 log.cc:1196] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/a3301648853a4a8f883f8ee2da7dcb39
I20260811 00:19:00.979496 19529 raft_consensus.cc:348] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.979769 19529 raft_consensus.cc:374] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:00.979864 19529 raft_consensus.cc:729] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec46bde515fc4de883748f69859ec5f5, State: Initialized, Role: FOLLOWER
I20260811 00:19:00.980159 19529 consensus_queue.cc:260] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.981345 19529 ts_tablet_manager.cc:1428] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 00:19:00.980749 19574 ts_tablet_manager.cc:1939] T a3301648853a4a8f883f8ee2da7dcb39 P b226d34f5c0049c9b03002a4fef0679c: Deleting consensus metadata
I20260811 00:19:00.984534 18600 catalog_manager.cc:4841] TS b226d34f5c0049c9b03002a4fef0679c (127.16.71.130:38721): tablet a3301648853a4a8f883f8ee2da7dcb39 (table test-table [id=3452fd1fe2304b2aa30515f621566025]) successfully deleted
I20260811 00:19:00.990422 19531 tablet_bootstrap.cc:492] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: No bootstrap required, opened a new log
I20260811 00:19:00.990545 19532 tablet_bootstrap.cc:492] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: No bootstrap required, opened a new log
I20260811 00:19:00.990612 19531 ts_tablet_manager.cc:1397] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260811 00:19:00.990729 19532 ts_tablet_manager.cc:1397] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.004s
I20260811 00:19:00.991366 19531 raft_consensus.cc:348] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.991568 19531 raft_consensus.cc:374] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:00.991660 19531 raft_consensus.cc:729] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796d5b9265164550a63e3dfd509478f0, State: Initialized, Role: FOLLOWER
I20260811 00:19:00.991626 19532 raft_consensus.cc:348] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.991901 19532 raft_consensus.cc:374] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:00.992017 19532 raft_consensus.cc:729] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b226d34f5c0049c9b03002a4fef0679c, State: Initialized, Role: FOLLOWER
I20260811 00:19:00.991984 19531 consensus_queue.cc:260] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.992300 19532 consensus_queue.cc:260] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:00.993011 19532 ts_tablet_manager.cc:1428] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:19:00.993975 19531 ts_tablet_manager.cc:1428] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260811 00:19:01.012876 19003 tablet.cc:2369] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:01.042976 19542 raft_consensus.cc:482] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:01.043417 19542 raft_consensus.cc:504] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.045821 19542 leader_election.cc:288] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b226d34f5c0049c9b03002a4fef0679c (127.16.71.130:38721), ec46bde515fc4de883748f69859ec5f5 (127.16.71.131:43831)
I20260811 00:19:01.054911 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.046226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:38721 (local address 127.16.71.129:42609)
0811 00:19:01.046886 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.046906 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.047121 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.047787 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.047793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.047826 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:01.048259 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.048272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.049939 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.049945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.052234 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.052250 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.054064 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.054070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.054236 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.054273 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.054374 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.054535 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":329,"threads_started":1}
I20260811 00:19:01.054983 19520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.046294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:42609 (local address 127.16.71.130:38721)
0811 00:19:01.046512 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.046519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.046992 (+ 473us) server_negotiation.cc:407] Connection header received
0811 00:19:01.047320 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.047326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.047412 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.047526 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:01.048488 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.049767 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.052431 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.053845 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.054013 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.054096 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.054430 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.054569 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:19:01.056290 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.047655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:43797 (local address 127.16.71.131:43831)
0811 00:19:01.047844 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.047850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.049849 (+ 1999us) server_negotiation.cc:407] Connection header received
0811 00:19:01.050196 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.050202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.050287 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.050384 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:01.051163 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.052241 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.053967 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.055144 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.055301 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.055378 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.055888 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.056038 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:19:01.056427 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.047062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43831 (local address 127.16.71.129:43797)
0811 00:19:01.049710 (+ 2648us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.049732 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.049975 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.050439 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.050444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.050476 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:01.050936 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.050950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.052436 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.052439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.053771 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.053786 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.055452 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.055459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.055650 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.055684 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.055825 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.055972 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2513,"thread_start_us":2276,"threads_started":1}
I20260811 00:19:01.057160 19407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86449d1b8d9348a3a82a2d3ef2c9d30f" candidate_uuid: "796d5b9265164550a63e3dfd509478f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec46bde515fc4de883748f69859ec5f5" is_pre_election: true
I20260811 00:19:01.057101 19164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86449d1b8d9348a3a82a2d3ef2c9d30f" candidate_uuid: "796d5b9265164550a63e3dfd509478f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b226d34f5c0049c9b03002a4fef0679c" is_pre_election: true
I20260811 00:19:01.057485 19407 raft_consensus.cc:2454] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 796d5b9265164550a63e3dfd509478f0 in term 0.
I20260811 00:19:01.057718 19164 raft_consensus.cc:2454] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 796d5b9265164550a63e3dfd509478f0 in term 0.
I20260811 00:19:01.058104 18786 leader_election.cc:302] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 796d5b9265164550a63e3dfd509478f0, ec46bde515fc4de883748f69859ec5f5; no voters:
I20260811 00:19:01.058578 19542 raft_consensus.cc:2781] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:01.058694 19542 raft_consensus.cc:482] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:01.058764 19542 raft_consensus.cc:3037] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:01.062007 19542 raft_consensus.cc:504] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.062937 19542 leader_election.cc:288] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [CANDIDATE]: Term 1 election: Requested vote from peers b226d34f5c0049c9b03002a4fef0679c (127.16.71.130:38721), ec46bde515fc4de883748f69859ec5f5 (127.16.71.131:43831)
I20260811 00:19:01.063510 19164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86449d1b8d9348a3a82a2d3ef2c9d30f" candidate_uuid: "796d5b9265164550a63e3dfd509478f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b226d34f5c0049c9b03002a4fef0679c"
I20260811 00:19:01.063764 19164 raft_consensus.cc:3037] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:01.063716 19407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86449d1b8d9348a3a82a2d3ef2c9d30f" candidate_uuid: "796d5b9265164550a63e3dfd509478f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec46bde515fc4de883748f69859ec5f5"
I20260811 00:19:01.063934 19407 raft_consensus.cc:3037] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:01.067131 19407 raft_consensus.cc:2454] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 796d5b9265164550a63e3dfd509478f0 in term 1.
I20260811 00:19:01.067894 18786 leader_election.cc:302] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 796d5b9265164550a63e3dfd509478f0, ec46bde515fc4de883748f69859ec5f5; no voters:
I20260811 00:19:01.068878 19542 raft_consensus.cc:2781] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:01.069046 19542 raft_consensus.cc:686] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 1 LEADER]: Becoming Leader. State: Replica: 796d5b9265164550a63e3dfd509478f0, State: Running, Role: LEADER
I20260811 00:19:01.069461 19542 consensus_queue.cc:237] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.070621 19164 raft_consensus.cc:2454] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 796d5b9265164550a63e3dfd509478f0 in term 1.
I20260811 00:19:01.083463 18620 catalog_manager.cc:5495] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 796d5b9265164550a63e3dfd509478f0 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "796d5b9265164550a63e3dfd509478f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:19:01 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407538, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:01.144362 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.127474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39707 (local address 127.0.0.1:45666)
0811 00:19:01.128264 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.128281 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.128432 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.128827 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.128830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.129570 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:01.129855 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.129866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.131387 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.131392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.132547 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.132563 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.133661 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.133666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.133805 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.134628 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:01.134660 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:01.137349 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:01.141585 (+ 4236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:01.141605 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:01.141611 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:01.142086 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:01.142554 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:01.142558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:01.142561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:01.142706 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:01.143439 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:01.143446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:01.143640 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.143791 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.144037 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":403,"threads_started":1}
I20260811 00:19:01.144708 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.127456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.16.71.190:39707)
0811 00:19:01.127704 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.127710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.128389 (+ 679us) server_negotiation.cc:407] Connection header received
0811 00:19:01.128557 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.128563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.128677 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.128800 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:01.130046 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.131204 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.132726 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.133445 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.133578 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.137592 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:01.137599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:01.137605 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:01.137645 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:01.141379 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.142230 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.142237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.142240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.142419 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.142813 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.142818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.142821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.143285 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:01.143468 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.143917 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.144218 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":85}
I20260811 00:19:01.151424 18620 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:45666:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:19:01.153227 18620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:01.161018 19132 tablet_service.cc:1467] Processing CreateTablet for tablet b38ea58b69da4d4082b1a7edcd21974d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a1cb25e110c48a8b93edd797dabfb25]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:19:01.161018 18878 tablet_service.cc:1467] Processing CreateTablet for tablet b38ea58b69da4d4082b1a7edcd21974d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a1cb25e110c48a8b93edd797dabfb25]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:19:01.161659 18878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b38ea58b69da4d4082b1a7edcd21974d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:01.161667 19132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b38ea58b69da4d4082b1a7edcd21974d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:01.168248 19531 tablet_bootstrap.cc:492] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: Bootstrap starting.
I20260811 00:19:01.168438 19375 tablet_service.cc:1467] Processing CreateTablet for tablet b38ea58b69da4d4082b1a7edcd21974d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a1cb25e110c48a8b93edd797dabfb25]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:19:01.169091 19375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b38ea58b69da4d4082b1a7edcd21974d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:01.170223 19531 tablet_bootstrap.cc:654] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:01.170245 19532 tablet_bootstrap.cc:492] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: Bootstrap starting.
I20260811 00:19:01.172194 19532 tablet_bootstrap.cc:654] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:01.176517 19529 tablet_bootstrap.cc:492] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: Bootstrap starting.
I20260811 00:19:01.178462 19529 tablet_bootstrap.cc:654] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:01.180208 19531 tablet_bootstrap.cc:492] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: No bootstrap required, opened a new log
I20260811 00:19:01.180483 19531 ts_tablet_manager.cc:1397] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260811 00:19:01.181205 19532 tablet_bootstrap.cc:492] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: No bootstrap required, opened a new log
I20260811 00:19:01.181417 19532 ts_tablet_manager.cc:1397] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260811 00:19:01.181691 19531 raft_consensus.cc:348] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.182034 19531 raft_consensus.cc:374] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:01.182178 19531 raft_consensus.cc:729] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796d5b9265164550a63e3dfd509478f0, State: Initialized, Role: FOLLOWER
I20260811 00:19:01.182309 19532 raft_consensus.cc:348] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.182583 19532 raft_consensus.cc:374] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:01.182686 19532 raft_consensus.cc:729] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b226d34f5c0049c9b03002a4fef0679c, State: Initialized, Role: FOLLOWER
I20260811 00:19:01.182799 19531 consensus_queue.cc:260] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.182979 19532 consensus_queue.cc:260] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.183439 19542 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.183597 19539 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.183837 19542 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:01.183887 19539 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:01.184902 19532 ts_tablet_manager.cc:1428] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 00:19:01.185199 19531 ts_tablet_manager.cc:1428] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 00:19:01.187955 19529 tablet_bootstrap.cc:492] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: No bootstrap required, opened a new log
I20260811 00:19:01.189028 19529 ts_tablet_manager.cc:1397] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260811 00:19:01.189905 19529 raft_consensus.cc:348] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.190135 19529 raft_consensus.cc:374] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:01.190222 19529 raft_consensus.cc:729] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec46bde515fc4de883748f69859ec5f5, State: Initialized, Role: FOLLOWER
I20260811 00:19:01.190523 19529 consensus_queue.cc:260] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.191017 19544 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.191336 19529 ts_tablet_manager.cc:1428] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 00:19:01.191488 19544 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:01.209986 19544 raft_consensus.cc:482] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:01.210253 19544 raft_consensus.cc:504] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.214820 19544 leader_election.cc:288] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 796d5b9265164550a63e3dfd509478f0 (127.16.71.129:33133), b226d34f5c0049c9b03002a4fef0679c (127.16.71.130:38721)
I20260811 00:19:01.230127 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.215718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:58773 (local address 127.16.71.129:33133)
0811 00:19:01.215961 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.215969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.222208 (+ 6239us) server_negotiation.cc:407] Connection header received
0811 00:19:01.222573 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.222579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.222666 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.222731 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:01.223542 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.224734 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.227444 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.228753 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.228918 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.229003 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.229643 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.229809 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 00:19:01.230930 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.215180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:33133 (local address 127.16.71.131:58773)
0811 00:19:01.222054 (+ 6874us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.222080 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.222362 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.222960 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.222966 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.223006 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:01.223315 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.223329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.224950 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.224955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.227202 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.227221 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.229175 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.229181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.229404 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.229444 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.230388 (+ 944us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.230557 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6625,"thread_start_us":6441,"threads_started":1}
I20260811 00:19:01.232542 18916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38ea58b69da4d4082b1a7edcd21974d" candidate_uuid: "ec46bde515fc4de883748f69859ec5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "796d5b9265164550a63e3dfd509478f0" is_pre_election: true
I20260811 00:19:01.232954 18916 raft_consensus.cc:2454] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec46bde515fc4de883748f69859ec5f5 in term 0.
I20260811 00:19:01.233572 19290 leader_election.cc:302] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 796d5b9265164550a63e3dfd509478f0, ec46bde515fc4de883748f69859ec5f5; no voters:
I20260811 00:19:01.234125 19544 raft_consensus.cc:2781] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:01.234243 19544 raft_consensus.cc:482] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:01.234342 19520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.222393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:58215 (local address 127.16.71.130:38721)
0811 00:19:01.222610 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.222616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.226776 (+ 4160us) server_negotiation.cc:407] Connection header received
0811 00:19:01.226954 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.226961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.227056 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.227163 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:01.228045 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.229647 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.231643 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.232888 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.233087 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.233168 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.233842 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.234019 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:19:01.234341 19544 raft_consensus.cc:3037] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:01.235316 19601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.222799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:38721 (local address 127.16.71.131:58215)
0811 00:19:01.226642 (+ 3843us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.226660 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.226807 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.227371 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.227376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.227407 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:01.227801 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.227816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.229834 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.229838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.231441 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.231458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.233408 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.233419 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.233611 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.233646 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.234815 (+ 1169us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.234987 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3680,"thread_start_us":3444,"threads_started":1}
I20260811 00:19:01.238256 19544 raft_consensus.cc:504] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.239300 19164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38ea58b69da4d4082b1a7edcd21974d" candidate_uuid: "ec46bde515fc4de883748f69859ec5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b226d34f5c0049c9b03002a4fef0679c" is_pre_election: true
I20260811 00:19:01.239631 19164 raft_consensus.cc:2454] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec46bde515fc4de883748f69859ec5f5 in term 0.
I20260811 00:19:01.239902 19544 leader_election.cc:288] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [CANDIDATE]: Term 1 election: Requested vote from peers 796d5b9265164550a63e3dfd509478f0 (127.16.71.129:33133), b226d34f5c0049c9b03002a4fef0679c (127.16.71.130:38721)
I20260811 00:19:01.241011 18916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38ea58b69da4d4082b1a7edcd21974d" candidate_uuid: "ec46bde515fc4de883748f69859ec5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "796d5b9265164550a63e3dfd509478f0"
I20260811 00:19:01.241276 18916 raft_consensus.cc:3037] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:01.241348 19164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38ea58b69da4d4082b1a7edcd21974d" candidate_uuid: "ec46bde515fc4de883748f69859ec5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b226d34f5c0049c9b03002a4fef0679c"
I20260811 00:19:01.241816 19164 raft_consensus.cc:3037] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:01.244658 19164 raft_consensus.cc:2454] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec46bde515fc4de883748f69859ec5f5 in term 1.
I20260811 00:19:01.244673 18916 raft_consensus.cc:2454] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec46bde515fc4de883748f69859ec5f5 in term 1.
I20260811 00:19:01.245783 19290 leader_election.cc:302] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b226d34f5c0049c9b03002a4fef0679c, ec46bde515fc4de883748f69859ec5f5; no voters:
I20260811 00:19:01.246165 19544 raft_consensus.cc:2781] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:01.246841 19544 raft_consensus.cc:686] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [term 1 LEADER]: Becoming Leader. State: Replica: ec46bde515fc4de883748f69859ec5f5, State: Running, Role: LEADER
I20260811 00:19:01.247249 19544 consensus_queue.cc:237] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } }
I20260811 00:19:01.251012 19604 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: TxnStatusTablet state changed. Reason: New leader ec46bde515fc4de883748f69859ec5f5. Latest consensus state: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.251444 19604 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:19:01.252302 19608 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:19:01.252660 19608 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:19:01.253106 18622 catalog_manager.cc:5495] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec46bde515fc4de883748f69859ec5f5 (127.16.71.131). New cstate: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } health_report { overall_health: HEALTHY } } }
I20260811 00:19:01.297333 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.288231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43831 (local address 127.0.0.1:57460)
0811 00:19:01.288815 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.288839 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.289010 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.290231 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.290236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.290251 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:19:01.290478 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.290491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.292140 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.292146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.293806 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.293826 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.295032 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.295038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.295243 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.295338 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:19:01.296642 (+ 1304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:19:01.296671 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.296762 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.296950 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363}
I20260811 00:19:01.298147 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.289340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.16.71.131:43831)
0811 00:19:01.289531 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.289536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.289554 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:19:01.289665 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.289671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.289819 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.289975 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:01.290690 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.291925 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.294004 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.294744 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.294882 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.295614 (+ 732us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:01.296414 (+ 800us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:01.296556 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.297680 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.297786 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:19:01.304052 18916 raft_consensus.cc:1261] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Refusing update from remote peer ec46bde515fc4de883748f69859ec5f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:01.304358 19164 raft_consensus.cc:1261] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Refusing update from remote peer ec46bde515fc4de883748f69859ec5f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:01.305254 19604 consensus_queue.cc:1035] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:01.305673 19544 consensus_queue.cc:1035] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:01.321681 19539 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: TxnStatusTablet state changed. Reason: New leader ec46bde515fc4de883748f69859ec5f5. Latest consensus state: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.322074 19539 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:01.324231 19542 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: TxnStatusTablet state changed. Reason: New leader ec46bde515fc4de883748f69859ec5f5. Latest consensus state: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.324589 19542 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:01.326618 19604 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.326939 19604 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:19:01.334630 19539 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.335003 19539 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P b226d34f5c0049c9b03002a4fef0679c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:01.341549 19544 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.341912 19544 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P ec46bde515fc4de883748f69859ec5f5: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:19:01.346279 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.338011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.16.71.129:33133)
0811 00:19:01.338190 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.338196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.338289 (+ 93us) server_negotiation.cc:407] Connection header received
0811 00:19:01.338584 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.338590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.338724 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.338892 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:01.339652 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.340827 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.342848 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.343466 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.343602 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.344290 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:01.345234 (+ 944us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:01.345379 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.345924 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.346026 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:19:01.349068 19542 tablet_replica.cc:439] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec46bde515fc4de883748f69859ec5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "796d5b9265164550a63e3dfd509478f0" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33133 } } peers { permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 } } peers { permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 } } }
I20260811 00:19:01.349437 19542 tablet_replica.cc:442] T b38ea58b69da4d4082b1a7edcd21974d P 796d5b9265164550a63e3dfd509478f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:01.374562 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.367608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:34921 (local address 127.16.71.131:43831)
0811 00:19:01.367836 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.367843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.369278 (+ 1435us) server_negotiation.cc:407] Connection header received
0811 00:19:01.369492 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.369496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.369564 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.369641 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:01.370186 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.371001 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.372504 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.373177 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.373302 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.373780 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:01.373895 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:01.373970 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.374106 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.374195 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:19:01.374631 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.367503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43831 (local address 127.16.71.129:34921)
0811 00:19:01.369153 (+ 1650us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.369170 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.369340 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.369675 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.369679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.369691 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:19:01.370022 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.370033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.371146 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.371152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.372340 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.372356 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.373384 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.373391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.373528 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.373595 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:19:01.373992 (+ 397us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:19:01.374004 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.374075 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.374246 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1474,"thread_start_us":1309,"threads_started":1}
I20260811 00:19:01.388420 19407 raft_consensus.cc:1261] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Refusing update from remote peer 796d5b9265164550a63e3dfd509478f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:01.389156 19542 consensus_queue.cc:1035] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec46bde515fc4de883748f69859ec5f5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:01.389361 19164 raft_consensus.cc:1261] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Refusing update from remote peer 796d5b9265164550a63e3dfd509478f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:01.390206 19542 consensus_queue.cc:1035] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b226d34f5c0049c9b03002a4fef0679c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:01.407812 19554 mvcc.cc:205] Tried to move back new op lower bound from 7317125289518247936 to 7317125288228728832. Current Snapshot: MvccSnapshot[applied={T|T < 7317125289518247936}]
I20260811 00:19:01.472522 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.455988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:44223 (local address 127.16.71.129:33133)
0811 00:19:01.456236 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.456242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.465558 (+ 9316us) server_negotiation.cc:407] Connection header received
0811 00:19:01.466089 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.466095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.466193 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.466263 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:01.467261 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.468299 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.470095 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.470661 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.470797 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.471283 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:01.471408 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:01.471520 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.472113 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.472206 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:19:01.473771 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.455860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:33133 (local address 127.16.71.131:44223)
0811 00:19:01.465429 (+ 9569us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:01.465449 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:01.465872 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:01.466476 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:01.466481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:01.466494 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:19:01.466988 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.467002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.468594 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.468600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.469931 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:01.469946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.470872 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.470880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:01.471019 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.471093 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:19:01.471949 (+ 856us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:19:01.471964 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 00:19:01.473136 (+ 1172us) client_negotiation.cc:240] Negotiation successful
0811 00:19:01.473304 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9339,"thread_start_us":9179,"threads_started":1}
I20260811 00:19:01.555315 18617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:01.555634 18617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:01.559865 18617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a90b7c9d6da544a58433306fd8fe4174: Sending DeleteTablet for 3 replicas of tablet 86449d1b8d9348a3a82a2d3ef2c9d30f
I20260811 00:19:01.561165 18878 tablet_service.cc:1514] Processing DeleteTablet for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:01 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:59892
I20260811 00:19:01.561307 19132 tablet_service.cc:1514] Processing DeleteTablet for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:01 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:60740
I20260811 00:19:01.561748 19375 tablet_service.cc:1514] Processing DeleteTablet for tablet 86449d1b8d9348a3a82a2d3ef2c9d30f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:01 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:57454
I20260811 00:19:01.598150 19634 tablet_replica.cc:330] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: stopping tablet replica
I20260811 00:19:01.600512 19634 raft_consensus.cc:2229] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:01.601001 19634 raft_consensus.cc:2258] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:01.601843 19630 tablet_replica.cc:330] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: stopping tablet replica
I20260811 00:19:01.602115 19630 raft_consensus.cc:2229] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:01.602802 19634 ts_tablet_manager.cc:1905] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:01.602800 19630 raft_consensus.cc:2258] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:01.603478 19633 tablet_replica.cc:330] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: stopping tablet replica
I20260811 00:19:01.603745 19633 raft_consensus.cc:2229] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:01.604048 19633 raft_consensus.cc:2258] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:01.605388 19633 ts_tablet_manager.cc:1905] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:01.609539 19630 ts_tablet_manager.cc:1905] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:01.614900 19634 ts_tablet_manager.cc:1918] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:01.617383 19634 log.cc:1196] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/86449d1b8d9348a3a82a2d3ef2c9d30f
I20260811 00:19:01.618148 19634 ts_tablet_manager.cc:1939] T 86449d1b8d9348a3a82a2d3ef2c9d30f P ec46bde515fc4de883748f69859ec5f5: Deleting consensus metadata
I20260811 00:19:01.618727 19630 ts_tablet_manager.cc:1918] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:01.619212 19630 log.cc:1196] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/86449d1b8d9348a3a82a2d3ef2c9d30f
I20260811 00:19:01.620826 19630 ts_tablet_manager.cc:1939] T 86449d1b8d9348a3a82a2d3ef2c9d30f P 796d5b9265164550a63e3dfd509478f0: Deleting consensus metadata
I20260811 00:19:01.620872 19633 ts_tablet_manager.cc:1918] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:01.621116 19633 log.cc:1196] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/86449d1b8d9348a3a82a2d3ef2c9d30f
I20260811 00:19:01.621677 19633 ts_tablet_manager.cc:1939] T 86449d1b8d9348a3a82a2d3ef2c9d30f P b226d34f5c0049c9b03002a4fef0679c: Deleting consensus metadata
I20260811 00:19:01.622299 18600 catalog_manager.cc:4841] TS 796d5b9265164550a63e3dfd509478f0 (127.16.71.129:33133): tablet 86449d1b8d9348a3a82a2d3ef2c9d30f (table test-table [id=3b46e80a333f49c99df265880415d0e6]) successfully deleted
I20260811 00:19:01.622961 18600 catalog_manager.cc:4841] TS b226d34f5c0049c9b03002a4fef0679c (127.16.71.130:38721): tablet 86449d1b8d9348a3a82a2d3ef2c9d30f (table test-table [id=3b46e80a333f49c99df265880415d0e6]) successfully deleted
I20260811 00:19:01.623421 18600 catalog_manager.cc:4841] TS ec46bde515fc4de883748f69859ec5f5 (127.16.71.131:43831): tablet 86449d1b8d9348a3a82a2d3ef2c9d30f (table test-table [id=3b46e80a333f49c99df265880415d0e6]) successfully deleted
Aug 11 00:19:01 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:01.643908 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.599222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.16.71.129:33133)
0811 00:19:01.599474 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.599482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.610914 (+ 11432us) server_negotiation.cc:407] Connection header received
0811 00:19:01.613847 (+ 2933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.613854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.613976 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.614086 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:01.622984 (+ 8898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.624217 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.626517 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.627263 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.627446 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.633537 (+ 6091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:01.633560 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:01.633581 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:01.633639 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:01.638760 (+ 5121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.640022 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.640034 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.640038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.640245 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.640823 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.640829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.640832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.641402 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:01.641771 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.643170 (+ 1399us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.643472 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 00:19:01.644968 18994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59910
I20260811 00:19:01.684479 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.657376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.16.71.190:39707)
0811 00:19:01.665351 (+ 7975us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.665358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.665388 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:19:01.665466 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.665472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.665562 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.665720 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:01.667942 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.669255 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.671442 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.672137 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.672277 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.675946 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:01.675955 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:01.675960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:01.676012 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:01.680140 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.681343 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.681350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.681354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.681605 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.682204 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.682210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.682213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.682628 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:01.682869 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.683544 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.683834 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7812,"thread_start_us":7665,"threads_started":1}
W20260811 00:19:01.685197 18617 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45672
I20260811 00:19:01.688246 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18758
I20260811 00:19:01.709934 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.690848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.16.71.129:33133)
0811 00:19:01.691122 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.691128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.693184 (+ 2056us) server_negotiation.cc:407] Connection header received
0811 00:19:01.693395 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.693401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.693495 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.693607 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:01.695170 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.696353 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.698302 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.699054 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.699204 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.702511 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:01.702518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:01.702522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:01.702569 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:01.706324 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.707271 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.707277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.707281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.707451 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.707900 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.707905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.707908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.708298 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:01.708588 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.709008 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.709268 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260811 00:19:01.710916 18994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59920
W20260811 00:19:01.711655 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18758: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:01.711885 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18758
I20260811 00:19:01.752702 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19006
Aug 11 00:19:01 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:19:01.770879 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.753508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.16.71.130:38721)
0811 00:19:01.754006 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.754011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.754026 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:19:01.754084 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.754088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.754151 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.754262 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:01.755500 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.756229 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.757563 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.757992 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.758124 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.763040 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:01.763063 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:01.763088 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:01.763148 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:01.767325 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.768280 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.768292 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.768296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.768520 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.769034 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.769040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.769043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.769549 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:01.769912 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.770201 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.770479 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":248,"threads_started":1}
W20260811 00:19:01.771656 19240 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60758
W20260811 00:19:01.772336 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19006: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:01.772516 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19006
I20260811 00:19:01.813232 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19254
Aug 11 00:19:01 dist-test-slave-f7th krb5kdc[18542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407540, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:19:01.836031 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:01.814006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57464 (local address 127.16.71.131:43831)
0811 00:19:01.814208 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:01.814215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:01.814315 (+ 100us) server_negotiation.cc:407] Connection header received
0811 00:19:01.814536 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:01.814541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:01.814614 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:01.814716 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:01.816707 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.817664 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.819617 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:01.820106 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:01.820226 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:01.826869 (+ 6643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:01.826896 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:01.826961 (+ 65us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:01.827027 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:01.831966 (+ 4939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.833320 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.833333 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.833337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.833573 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:01.834073 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:01.834079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:01.834083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:01.834602 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:01.834999 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:01.835333 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 00:19:01.835603 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 00:19:01.836911 19487 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57464
W20260811 00:19:01.837741 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19254: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:01.837961 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19254
I20260811 00:19:01.872236 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18565
I20260811 00:19:01.873558 18687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:01.992502 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18565
2026-08-11T00:19:02Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5134 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:02 dist-test-slave-f7th krb5kdc[19654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:02 dist-test-slave-f7th krb5kdc[19654](info): set up 2 sockets
Aug 11 00:19:02 dist-test-slave-f7th krb5kdc[19654](info): commencing operation
krb5kdc: starting...
W20260811 00:19:03.287508 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:03 dist-test-slave-f7th krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:19:03.522190 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39881
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:39881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:03.879868 19675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:03.880218 19675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:03.880335 19675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:03.888370 19675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:03.888525 19675 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:19:03.888576 19675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:03.888700 19675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:03.888773 19675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:03.888875 19675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:03.899694 19675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:39881
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:03.901286 19675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:03.902827 19675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:03.915493 19681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:03.915568 19682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:03.915525 19688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:03.916227 19675 server_base.cc:1034] running on GCE node
I20260811 00:19:03.917169 19675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:03.918748 19675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:03.920055 19675 hybrid_clock.cc:648] HybridClock initialized: now 1786407543919972 us; error 59 us; skew 500 ppm
I20260811 00:19:03.920464 19675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:03 dist-test-slave-f7th krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407543, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:03.924055 19675 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:19:03.926025 19675 webserver.cc:460] Webserver started at http://127.16.71.190:44715/ using document root <none> and password file <none>
I20260811 00:19:03.926923 19675 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:03.927035 19675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:03.927395 19675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:03.931684 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "26e2fd4ed0e041228a6454b9cda897ce"
format_stamp: "Formatted at 2026-08-11 00:19:03 on dist-test-slave-f7th"
server_key: "54e136142ea5ea43818426a8eca7bee7"
server_key_iv: "8f0a6adc58197ff6e3c022be81647335"
server_key_version: "encryptionkey@0"
I20260811 00:19:03.932797 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "26e2fd4ed0e041228a6454b9cda897ce"
format_stamp: "Formatted at 2026-08-11 00:19:03 on dist-test-slave-f7th"
server_key: "54e136142ea5ea43818426a8eca7bee7"
server_key_iv: "8f0a6adc58197ff6e3c022be81647335"
server_key_version: "encryptionkey@0"
I20260811 00:19:03.939499 19675 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:19:03.944762 19703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:03.947010 19675 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 00:19:03.947260 19675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "26e2fd4ed0e041228a6454b9cda897ce"
format_stamp: "Formatted at 2026-08-11 00:19:03 on dist-test-slave-f7th"
server_key: "54e136142ea5ea43818426a8eca7bee7"
server_key_iv: "8f0a6adc58197ff6e3c022be81647335"
server_key_version: "encryptionkey@0"
I20260811 00:19:03.947536 19675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:03.973696 19675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:03.978808 19675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:03.979102 19675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:04.006711 19675 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:39881
I20260811 00:19:04.006768 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:39881 every 8 connection(s)
I20260811 00:19:04.008335 19675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:04.013307 19813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:04.017310 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19675
I20260811 00:19:04.017584 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:04.018474 16670 external_mini_cluster.cc:1428] Setting key 7ecb1c3e048fc069abae0c82c68d94cd
I20260811 00:19:04.030161 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Bootstrap starting.
I20260811 00:19:04.034770 19813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:04.036379 19813 log.cc:824] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:19:04 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:04.040195 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: No bootstrap required, opened a new log
I20260811 00:19:04.046794 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:04.021162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51336 (local address 127.16.71.190:39881)
0811 00:19:04.022127 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:04.022141 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:04.022299 (+ 158us) server_negotiation.cc:407] Connection header received
0811 00:19:04.024559 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:04.024593 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:04.025349 (+ 756us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:04.025817 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:04.027967 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.029606 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.031557 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.032485 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.032728 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:04.038323 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:04.038333 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:04.038352 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:04.038394 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:04.041920 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:04.042801 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:04.042813 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:04.042820 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:04.043040 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:04.043465 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:04.043469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:04.043471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:04.044121 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:04.044425 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:04.045112 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 00:19:04.045619 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":428,"threads_started":1}
I20260811 00:19:04.049142 19813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e2fd4ed0e041228a6454b9cda897ce" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39881 } }
I20260811 00:19:04.049554 19813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:04.049676 19813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e2fd4ed0e041228a6454b9cda897ce, State: Initialized, Role: FOLLOWER
I20260811 00:19:04.050336 19813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e2fd4ed0e041228a6454b9cda897ce" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39881 } }
I20260811 00:19:04.050591 19813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:04.050724 19813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:04.050900 19813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:04.053640 19813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e2fd4ed0e041228a6454b9cda897ce" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39881 } }
I20260811 00:19:04.054395 19813 leader_election.cc:302] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26e2fd4ed0e041228a6454b9cda897ce; no voters:
I20260811 00:19:04.055022 19813 leader_election.cc:288] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:04.055150 19819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:04.055846 19819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [term 1 LEADER]: Becoming Leader. State: Replica: 26e2fd4ed0e041228a6454b9cda897ce, State: Running, Role: LEADER
I20260811 00:19:04.056473 19819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e2fd4ed0e041228a6454b9cda897ce" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39881 } }
I20260811 00:19:04.057346 19813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:04.059777 19822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26e2fd4ed0e041228a6454b9cda897ce. Latest consensus state: current_term: 1 leader_uuid: "26e2fd4ed0e041228a6454b9cda897ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e2fd4ed0e041228a6454b9cda897ce" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39881 } } }
I20260811 00:19:04.059708 19821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26e2fd4ed0e041228a6454b9cda897ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e2fd4ed0e041228a6454b9cda897ce" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39881 } } }
I20260811 00:19:04.060204 19821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:04.060912 19822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:04.061964 19832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:04.068681 19832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:04.079634 19832 catalog_manager.cc:1347] Generated new cluster ID: a481277b9cdc44c7ac90a061ba246be1
I20260811 00:19:04.079782 19832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:04.093905 19832 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:04.095363 19832 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:04.106374 19832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Generated new TSK 0
I20260811 00:19:04.107293 19832 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:19:04.172984 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39881
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:04.532140 19859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:04.532459 19859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:04.532603 19859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:04.540300 19859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:04.540454 19859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:04.540581 19859 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:19:04.540681 19859 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:19:04.540788 19859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:04.551188 19859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:39881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:04.552670 19859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:04.554163 19859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:04.568693 19868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:04.568717 19869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:04.568656 19876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:04.569885 19859 server_base.cc:1034] running on GCE node
I20260811 00:19:04.570606 19859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:04.571641 19859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:04.572852 19859 hybrid_clock.cc:648] HybridClock initialized: now 1786407544572787 us; error 55 us; skew 500 ppm
I20260811 00:19:04.573220 19859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:04 dist-test-slave-f7th krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407544, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:04.581434 19859 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:19:04.583340 19859 webserver.cc:460] Webserver started at http://127.16.71.129:42411/ using document root <none> and password file <none>
I20260811 00:19:04.584118 19859 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:04.584251 19859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:04.584594 19859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:04.588012 19859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "b700a56343fe413b9cfb535ace4de46f"
format_stamp: "Formatted at 2026-08-11 00:19:04 on dist-test-slave-f7th"
server_key: "b62426c3b523ef65637b5a87cf2bf9f6"
server_key_iv: "127367b9e61ab5e02fe5df9675e36aaa"
server_key_version: "encryptionkey@0"
I20260811 00:19:04.588979 19859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "b700a56343fe413b9cfb535ace4de46f"
format_stamp: "Formatted at 2026-08-11 00:19:04 on dist-test-slave-f7th"
server_key: "b62426c3b523ef65637b5a87cf2bf9f6"
server_key_iv: "127367b9e61ab5e02fe5df9675e36aaa"
server_key_version: "encryptionkey@0"
I20260811 00:19:04.594893 19859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:19:04.599038 19890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:04.600880 19859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:04.601094 19859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "b700a56343fe413b9cfb535ace4de46f"
format_stamp: "Formatted at 2026-08-11 00:19:04 on dist-test-slave-f7th"
server_key: "b62426c3b523ef65637b5a87cf2bf9f6"
server_key_iv: "127367b9e61ab5e02fe5df9675e36aaa"
server_key_version: "encryptionkey@0"
I20260811 00:19:04.601331 19859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:04.623883 19859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:04.628697 19859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:04.629007 19859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:04.631897 19859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:04.632022 19859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:04.632145 19859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:04.632225 19859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:04 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407544, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:04.656172 19905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:04.632740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:51389 (local address 127.16.71.190:39881)
0811 00:19:04.633230 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:04.633236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:04.634561 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 00:19:04.637492 (+ 2931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:04.637499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:04.637609 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:04.637727 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:04.640133 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.641137 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.643526 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.644188 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.644343 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:04.649346 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:04.649352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:04.649362 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:04.649400 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:04.652355 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:04.653618 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:04.653622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:04.653624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:04.653759 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:04.654252 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:04.654256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:04.654258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:04.654552 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:04.654703 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:04.655537 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 00:19:04.655789 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":237,"threads_started":1}
I20260811 00:19:04.657560 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:04.633423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39881 (local address 127.16.71.129:51389)
0811 00:19:04.634342 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:04.634405 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:04.637046 (+ 2641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:04.638085 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:04.638097 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:04.638925 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:04.639929 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:04.639956 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.641303 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.641310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:04.643348 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:04.643370 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.644477 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.644485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:04.644838 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:04.645745 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:04.645796 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:04.649064 (+ 3268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:04.652597 (+ 3533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:04.652606 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:04.652655 (+ 49us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:04.653411 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:04.653882 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:04.653887 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:04.653891 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:04.654119 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:04.654695 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:04.654703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:04.655120 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 00:19:04.655395 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 00:19:04.655965 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":340,"threads_started":1}
I20260811 00:19:04.680797 19905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:04.663896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37425 (local address 127.16.71.190:39881)
0811 00:19:04.664126 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:04.664133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:04.664838 (+ 705us) server_negotiation.cc:407] Connection header received
0811 00:19:04.665057 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:04.665063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:04.665137 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:04.665225 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:04.666576 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.667497 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.669565 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.670248 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.670392 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:04.673484 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:04.673492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:04.673498 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:04.673544 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:04.677221 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:04.678480 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:04.678487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:04.678490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:04.678681 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:04.679130 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:04.679134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:04.679136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:04.679465 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:04.679641 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:04.680195 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 00:19:04.680428 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:19:04.680779 19859 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:41783
I20260811 00:19:04.680850 20111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:41783 every 8 connection(s)
I20260811 00:19:04.681157 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:04.664007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39881 (local address 127.16.71.129:37425)
0811 00:19:04.664720 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:04.664737 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:04.664919 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:04.665269 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:04.665274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:04.665991 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:04.666367 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:04.666379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.667673 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.667678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:04.669327 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:04.669343 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:04.670699 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:04.670708 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:04.670912 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:04.671798 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:04.671822 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:04.673138 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:04.677498 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:04.677505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:04.677510 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:04.678259 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:04.678818 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:04.678824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:04.678828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:04.678999 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:04.679619 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:04.679624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:04.679873 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 00:19:04.680048 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 00:19:04.680494 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":325,"threads_started":1}
I20260811 00:19:04.682441 19859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:04.684775 20112 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39881
I20260811 00:19:04.685026 20112 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:04.685726 20112 heartbeater.cc:511] Master 127.16.71.190:39881 requested a full tablet report, sending...
I20260811 00:19:04.687803 19733 ts_manager.cc:194] Registered new tserver with Master: b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783)
I20260811 00:19:04.689147 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19859
I20260811 00:19:04.689358 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:04.689767 19733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:51389
I20260811 00:19:04.690160 16670 external_mini_cluster.cc:1428] Setting key 9c0e0ce99f09c54f495170ade501d3dc
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:19:04.739992 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39881
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:05.104889 20122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:05.105252 20122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:05.105439 20122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:05.113843 20122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:05.114019 20122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:05.114161 20122 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:19:05.114234 20122 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:19:05.114344 20122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:05.125512 20122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:39881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:05.127274 20122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:05.129024 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:05.144328 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:05.144289 20138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:05.144352 20130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:05.145236 20122 server_base.cc:1034] running on GCE node
I20260811 00:19:05.146013 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:05.147150 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:05.148408 20122 hybrid_clock.cc:648] HybridClock initialized: now 1786407545148324 us; error 55 us; skew 500 ppm
I20260811 00:19:05.148793 20122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:05 dist-test-slave-f7th krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407545, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:05.152311 20122 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:19:05.154151 20122 webserver.cc:460] Webserver started at http://127.16.71.130:42561/ using document root <none> and password file <none>
I20260811 00:19:05.154901 20122 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:05.155048 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:05.155368 20122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:05.158624 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "0021d6ebb63741a2b3204dfe69b32e33"
format_stamp: "Formatted at 2026-08-11 00:19:05 on dist-test-slave-f7th"
server_key: "f71493720a0cca10d5387a7b0b32d25d"
server_key_iv: "f4bc474eab00f7ffb5e5e357d09427d7"
server_key_version: "encryptionkey@0"
I20260811 00:19:05.159492 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "0021d6ebb63741a2b3204dfe69b32e33"
format_stamp: "Formatted at 2026-08-11 00:19:05 on dist-test-slave-f7th"
server_key: "f71493720a0cca10d5387a7b0b32d25d"
server_key_iv: "f4bc474eab00f7ffb5e5e357d09427d7"
server_key_version: "encryptionkey@0"
I20260811 00:19:05.165756 20122 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 00:19:05.169837 20154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:05.171442 20122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:19:05.171651 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "0021d6ebb63741a2b3204dfe69b32e33"
format_stamp: "Formatted at 2026-08-11 00:19:05 on dist-test-slave-f7th"
server_key: "f71493720a0cca10d5387a7b0b32d25d"
server_key_iv: "f4bc474eab00f7ffb5e5e357d09427d7"
server_key_version: "encryptionkey@0"
I20260811 00:19:05.171945 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:05.193845 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:05.198657 20122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:05.198971 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:05.202313 20122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:05.202463 20122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:05.202594 20122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:05.202684 20122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:05 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407545, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:05.228761 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.202651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:54961 (local address 127.16.71.190:39881)
0811 00:19:05.203236 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.203243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.204554 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 00:19:05.206914 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.206919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.207016 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.207119 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:05.209490 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.210459 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.213712 (+ 3253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.214327 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.214508 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.220152 (+ 5644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:05.220158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:05.220161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:05.220203 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:05.224143 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.225279 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.225286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.225290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.225498 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.226202 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.226207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.226210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.226533 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:05.226708 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.227755 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.228304 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":357,"threads_started":1}
I20260811 00:19:05.230077 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.203429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39881 (local address 127.16.71.130:54961)
0811 00:19:05.204374 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:05.204419 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:05.206556 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:05.207393 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:05.207403 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:05.208234 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:05.209269 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.209294 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.210691 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.210699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.213487 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.213512 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.214677 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.214690 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.215060 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.215963 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:05.215997 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:05.219810 (+ 3813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:05.224377 (+ 4567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.224388 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.224406 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.225083 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.225647 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.225653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.225657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.226048 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.226826 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:05.226836 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:05.227322 (+ 486us) client_negotiation.cc:769] Sending connection context
0811 00:19:05.227605 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 00:19:05.228683 (+ 1078us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":377,"threads_started":1}
I20260811 00:19:05.254103 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.237477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39881 (local address 127.16.71.130:42531)
0811 00:19:05.238243 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:05.238267 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:05.238504 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:05.238943 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:05.238948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:05.239859 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:05.240344 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.240362 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.241842 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.241852 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.243879 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.243908 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.244901 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.244911 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.245119 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.245846 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:05.245872 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:05.246766 (+ 894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:05.250083 (+ 3317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.250090 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.250094 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.250681 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.251243 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.251248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.251252 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.251450 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.252548 (+ 1098us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:05.252554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:05.252841 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 00:19:05.253047 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 00:19:05.253607 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":385,"threads_started":1}
I20260811 00:19:05.254544 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.237449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:42531 (local address 127.16.71.190:39881)
0811 00:19:05.237682 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.237689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.238461 (+ 772us) server_negotiation.cc:407] Connection header received
0811 00:19:05.238691 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.238696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.238785 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.238888 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:05.240606 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.241632 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.244121 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.244606 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.244796 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.247027 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:05.247033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:05.247036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:05.247078 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:05.249873 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.250879 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.250885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.250888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.251081 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.251623 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.251631 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.251635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.252248 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:05.252440 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.253309 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.253764 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:19:05.266376 20122 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:34947
I20260811 00:19:05.266407 20370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:34947 every 8 connection(s)
I20260811 00:19:05.267814 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:05.270401 20371 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39881
I20260811 00:19:05.270678 20371 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:05.271421 20371 heartbeater.cc:511] Master 127.16.71.190:39881 requested a full tablet report, sending...
I20260811 00:19:05.273228 19733 ts_manager.cc:194] Registered new tserver with Master: 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947)
I20260811 00:19:05.274083 19733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:54961
I20260811 00:19:05.276115 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 20122
I20260811 00:19:05.276294 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:05.276836 16670 external_mini_cluster.cc:1428] Setting key dd3eb9582026e03aff1250512118f877
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:19:05.328142 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39881
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:19:05.693418 20112 heartbeater.cc:503] Master 127.16.71.190:39881 was elected leader, sending a full tablet report...
W20260811 00:19:05.693356 20378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:05.693665 20378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:05.693830 20378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:05.702005 20378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:05.702152 20378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:05.702279 20378 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:19:05.702344 20378 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:19:05.702450 20378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:05.713120 20378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:39881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:05.714637 20378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:05.716089 20378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:05.731065 20385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:05.730970 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:05.731033 20388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:05.732465 20378 server_base.cc:1034] running on GCE node
I20260811 00:19:05.733330 20378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:05.734407 20378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:05.735677 20378 hybrid_clock.cc:648] HybridClock initialized: now 1786407545735614 us; error 52 us; skew 500 ppm
I20260811 00:19:05.736104 20378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:05 dist-test-slave-f7th krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407545, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:05.739550 20378 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:19:05.741400 20378 webserver.cc:460] Webserver started at http://127.16.71.131:34407/ using document root <none> and password file <none>
I20260811 00:19:05.742184 20378 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:05.742324 20378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:05.742666 20378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:05.746330 20378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "a50d901bb3354af58d42f9eadb9b2113"
format_stamp: "Formatted at 2026-08-11 00:19:05 on dist-test-slave-f7th"
server_key: "4fd03abfee5423a3ab558062a6b32240"
server_key_iv: "390e2167537ac51e6ee3537b9b135ff6"
server_key_version: "encryptionkey@0"
I20260811 00:19:05.747243 20378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "a50d901bb3354af58d42f9eadb9b2113"
format_stamp: "Formatted at 2026-08-11 00:19:05 on dist-test-slave-f7th"
server_key: "4fd03abfee5423a3ab558062a6b32240"
server_key_iv: "390e2167537ac51e6ee3537b9b135ff6"
server_key_version: "encryptionkey@0"
I20260811 00:19:05.753419 20378 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:19:05.757658 20403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:05.759425 20378 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:19:05.759641 20378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "a50d901bb3354af58d42f9eadb9b2113"
format_stamp: "Formatted at 2026-08-11 00:19:05 on dist-test-slave-f7th"
server_key: "4fd03abfee5423a3ab558062a6b32240"
server_key_iv: "390e2167537ac51e6ee3537b9b135ff6"
server_key_version: "encryptionkey@0"
I20260811 00:19:05.759876 20378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:05.774725 20378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:05.780037 20378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:05.780334 20378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:05.783427 20378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:05.783563 20378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:05.783694 20378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:05.783772 20378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:05 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407545, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:05.815111 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.785420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:49845 (local address 127.16.71.190:39881)
0811 00:19:05.786903 (+ 1483us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.786913 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.789489 (+ 2576us) server_negotiation.cc:407] Connection header received
0811 00:19:05.793244 (+ 3755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.793253 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.793436 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.793556 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:05.797598 (+ 4042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.798395 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.801263 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.802021 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.802200 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.806521 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:05.806539 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:05.806545 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:05.806601 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:05.810592 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.811818 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.811825 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.811830 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.812031 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.812849 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.812853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.812855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.813281 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:05.813447 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.814381 (+ 934us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.814690 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1209,"thread_start_us":2989,"threads_started":1}
I20260811 00:19:05.816443 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.785169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39881 (local address 127.16.71.131:49845)
0811 00:19:05.789245 (+ 4076us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:05.789315 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:05.792714 (+ 3399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:05.794342 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:05.794358 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:05.795450 (+ 1092us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:05.797244 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.797288 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.798619 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.798629 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.801010 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.801036 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.802308 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.802317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.802591 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.803429 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:05.803468 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:05.806156 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:05.810863 (+ 4707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.810873 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.810890 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.811636 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.812349 (+ 713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.812355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.812358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.812703 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.813448 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:05.813457 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:05.813902 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 00:19:05.814227 (+ 325us) client_negotiation.cc:240] Negotiation successful
0811 00:19:05.814862 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3676,"thread_start_us":3375,"threads_started":1}
I20260811 00:19:05.841092 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.824180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:59531 (local address 127.16.71.190:39881)
0811 00:19:05.824415 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.824419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.825294 (+ 875us) server_negotiation.cc:407] Connection header received
0811 00:19:05.825515 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.825520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.825595 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.825686 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:05.827047 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.828072 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.830241 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.830818 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.830996 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.834082 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:05.834088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:05.834092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:05.834136 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:05.837426 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.838500 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.838505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.838507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.838674 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.839220 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.839226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.839230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.839605 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:05.839751 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.840384 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.840703 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:19:05.841298 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.824152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39881 (local address 127.16.71.131:59531)
0811 00:19:05.825150 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:05.825173 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:05.825358 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:05.825747 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:05.825751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:05.826345 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:05.826860 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.826875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.828255 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.828259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.829961 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.829987 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.831139 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.831166 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.831383 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.832267 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:05.832294 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:05.833710 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:05.837642 (+ 3932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.837649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.837653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.838313 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.838819 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:05.838824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:05.838828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:05.839044 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:05.839738 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:05.839745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:05.840032 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 00:19:05.840216 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 00:19:05.840727 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":608,"threads_started":1}
I20260811 00:19:05.850976 20378 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:43745
I20260811 00:19:05.851045 20617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:43745 every 8 connection(s)
I20260811 00:19:05.852437 20378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:05.853814 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 20378
I20260811 00:19:05.854210 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:05.854835 16670 external_mini_cluster.cc:1428] Setting key 65fa1095c47e0989817faa488c99086a
I20260811 00:19:05.855690 20620 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39881
I20260811 00:19:05.855981 20620 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:05.856962 20620 heartbeater.cc:511] Master 127.16.71.190:39881 requested a full tablet report, sending...
I20260811 00:19:05.859750 19733 ts_manager.cc:194] Registered new tserver with Master: a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745)
I20260811 00:19:05.860805 19733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:49845
I20260811 00:19:05.872824 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:19:05 dist-test-slave-f7th krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:19:05 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:05.930943 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.915198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.16.71.190:39881)
0811 00:19:05.915404 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.915410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.915721 (+ 311us) server_negotiation.cc:407] Connection header received
0811 00:19:05.916014 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.916019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.916093 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.916190 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:05.918003 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.918852 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.920328 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.920833 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.920987 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.925216 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:05.925222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:05.925225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:05.925267 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:05.927998 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.928898 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.928903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.928905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.929095 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:05.929498 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:05.929501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:05.929503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:05.929802 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:05.930034 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.930467 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.930668 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:19:05.935886 19733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:05.943269 19733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:05.983623 20644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.956680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34947 (local address 127.0.0.1:53058)
0811 00:19:05.970934 (+ 14254us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:05.971020 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:05.971490 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:05.974212 (+ 2722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:05.974220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:05.974259 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:05.975206 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.975266 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.977343 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.977349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.980161 (+ 2812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.980177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.982177 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.982183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.982605 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.982656 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 00:19:05.982844 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 00:19:05.983124 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13683,"mutex_wait_us":106,"thread_start_us":18722,"threads_started":1}
I20260811 00:19:05.985385 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.957150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.16.71.130:34947)
0811 00:19:05.967930 (+ 10780us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.967941 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.971896 (+ 3955us) server_negotiation.cc:407] Connection header received
0811 00:19:05.972067 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.972077 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.972480 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.972975 (+ 495us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:05.975502 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.977169 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.980385 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.981907 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.982044 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.982302 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.984556 (+ 2254us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.984866 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10589,"thread_start_us":10439,"threads_started":1}
I20260811 00:19:05.987854 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.957680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.16.71.131:43745)
0811 00:19:05.977858 (+ 20178us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.977864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.977894 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:19:05.977984 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.977989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.978175 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.978965 (+ 790us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:05.979771 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.981487 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.984005 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.986462 (+ 2457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.986605 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.987049 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.987304 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.987503 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20029,"thread_start_us":19815,"threads_started":1}
I20260811 00:19:05.987839 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.956933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43745 (local address 127.0.0.1:45582)
0811 00:19:05.975149 (+ 18216us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:05.975179 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:05.975721 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:05.978443 (+ 2722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:05.978447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:05.978475 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:05.979466 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.979485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.981660 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.981666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.983738 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.983753 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.986762 (+ 3009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.986769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.987049 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.987087 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:19:05.987206 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 00:19:05.987367 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18029,"mutex_wait_us":1,"thread_start_us":16273,"threads_started":1}
I20260811 00:19:05.988801 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.958205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.16.71.129:41783)
0811 00:19:05.977184 (+ 18979us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:05.977192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:05.977225 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:19:05.977578 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:05.977588 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:05.977875 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:05.978233 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:05.979131 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.980420 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.982832 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.985157 (+ 2325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.985331 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.985856 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:05.987980 (+ 2124us) server_negotiation.cc:299] Negotiation successful
0811 00:19:05.988184 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18671,"thread_start_us":18780,"threads_started":1}
I20260811 00:19:05.989252 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:05.957535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41783 (local address 127.0.0.1:47596)
0811 00:19:05.976408 (+ 18873us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:05.976430 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:05.977343 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:05.978231 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:05.978236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:05.978274 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:05.978907 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.978923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.980556 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.980561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.982600 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:05.982617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:05.985460 (+ 2843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:05.985469 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:05.987425 (+ 1956us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:05.987459 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:19:05.987566 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 00:19:05.987706 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18716,"mutex_wait_us":84,"spinlock_wait_cycles":2816,"thread_start_us":18580,"threads_started":1}
I20260811 00:19:05.991627 20247 tablet_service.cc:1467] Processing CreateTablet for tablet ff1e3970d3db49568ef608ced31a6112 (DEFAULT_TABLE table=test-table [id=60eccac28de545f99cc0fd46dc5e0467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:05.993961 20502 tablet_service.cc:1467] Processing CreateTablet for tablet ff1e3970d3db49568ef608ced31a6112 (DEFAULT_TABLE table=test-table [id=60eccac28de545f99cc0fd46dc5e0467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:05.995517 20247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1e3970d3db49568ef608ced31a6112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:05.995741 20502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1e3970d3db49568ef608ced31a6112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:05.997824 19980 tablet_service.cc:1467] Processing CreateTablet for tablet ff1e3970d3db49568ef608ced31a6112 (DEFAULT_TABLE table=test-table [id=60eccac28de545f99cc0fd46dc5e0467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:06.000317 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1e3970d3db49568ef608ced31a6112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:06.017647 20653 tablet_bootstrap.cc:492] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Bootstrap starting.
I20260811 00:19:06.022430 20654 tablet_bootstrap.cc:492] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Bootstrap starting.
I20260811 00:19:06.025266 20653 tablet_bootstrap.cc:654] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.026474 20655 tablet_bootstrap.cc:492] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Bootstrap starting.
I20260811 00:19:06.026770 20653 log.cc:824] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:06.029999 20654 tablet_bootstrap.cc:654] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.032362 20654 log.cc:824] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:06.033716 20655 tablet_bootstrap.cc:654] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.035382 20655 log.cc:824] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:06.035940 20653 tablet_bootstrap.cc:492] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: No bootstrap required, opened a new log
I20260811 00:19:06.036310 20653 ts_tablet_manager.cc:1397] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.006s
I20260811 00:19:06.039012 20655 tablet_bootstrap.cc:492] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: No bootstrap required, opened a new log
I20260811 00:19:06.039320 20655 ts_tablet_manager.cc:1397] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260811 00:19:06.039333 20654 tablet_bootstrap.cc:492] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: No bootstrap required, opened a new log
I20260811 00:19:06.039623 20654 ts_tablet_manager.cc:1397] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.000s
I20260811 00:19:06.047797 20655 raft_consensus.cc:348] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.047816 20653 raft_consensus.cc:348] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.048251 20655 raft_consensus.cc:374] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.048297 20653 raft_consensus.cc:374] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.048362 20655 raft_consensus.cc:729] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b700a56343fe413b9cfb535ace4de46f, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.048400 20653 raft_consensus.cc:729] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d901bb3354af58d42f9eadb9b2113, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.049144 20655 consensus_queue.cc:260] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.049162 20653 consensus_queue.cc:260] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.050670 20654 raft_consensus.cc:348] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.051128 20655 ts_tablet_manager.cc:1428] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 00:19:06.051225 20654 raft_consensus.cc:374] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.051326 20654 raft_consensus.cc:729] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.052171 20620 heartbeater.cc:503] Master 127.16.71.190:39881 was elected leader, sending a full tablet report...
I20260811 00:19:06.052205 20654 consensus_queue.cc:260] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.052968 20653 ts_tablet_manager.cc:1428] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Time spent starting tablet: real 0.017s user 0.009s sys 0.008s
I20260811 00:19:06.054523 20371 heartbeater.cc:503] Master 127.16.71.190:39881 was elected leader, sending a full tablet report...
I20260811 00:19:06.055547 20654 ts_tablet_manager.cc:1428] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
W20260811 00:19:06.105075 20621 tablet.cc:2369] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:06.159646 20664 raft_consensus.cc:482] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:06.160043 20664 raft_consensus.cc:504] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.162027 20664 leader_election.cc:288] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745), b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783)
I20260811 00:19:06.167699 20661 raft_consensus.cc:482] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:06.168052 20661 raft_consensus.cc:504] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.170120 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.162616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:52059 (local address 127.16.71.131:43745)
0811 00:19:06.162841 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.162848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.163483 (+ 635us) server_negotiation.cc:407] Connection header received
0811 00:19:06.163722 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.163727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.163801 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.163879 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:06.164650 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.165715 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.167740 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.168997 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.169186 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.169267 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.169693 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.169855 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:19:06.170291 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.162576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:56319 (local address 127.16.71.129:41783)
0811 00:19:06.162801 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.162808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.163929 (+ 1121us) server_negotiation.cc:407] Connection header received
0811 00:19:06.164207 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.164213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.164303 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.164407 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:06.165185 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.166055 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.168022 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.169331 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.169481 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.169552 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.169923 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.170060 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:19:06.170324 20661 leader_election.cc:288] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947), a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745)
I20260811 00:19:06.170552 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.162565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43745 (local address 127.16.71.130:52059)
0811 00:19:06.163357 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.163374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.163571 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.163928 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.163933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.164028 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:06.164414 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.164428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.165859 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.165865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.167547 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.167564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.169239 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.169246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.169424 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.169474 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.169637 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.169792 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":3,"spinlock_wait_cycles":29056,"thread_start_us":445,"threads_started":1}
I20260811 00:19:06.170955 20666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.163020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41783 (local address 127.16.71.130:56319)
0811 00:19:06.163813 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.163829 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.163969 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.164459 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.164463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.164486 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:06.165015 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.165029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.166197 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.166202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.167803 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.167819 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.169552 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.169559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.169730 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.169763 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.170480 (+ 717us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.170629 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":470,"threads_started":1}
I20260811 00:19:06.172235 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1e3970d3db49568ef608ced31a6112" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f" is_pre_election: true
I20260811 00:19:06.172693 20544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1e3970d3db49568ef608ced31a6112" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50d901bb3354af58d42f9eadb9b2113" is_pre_election: true
I20260811 00:19:06.172873 20027 raft_consensus.cc:2454] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 0.
I20260811 00:19:06.173382 20544 raft_consensus.cc:2454] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 0.
I20260811 00:19:06.173698 20158 leader_election.cc:302] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0021d6ebb63741a2b3204dfe69b32e33, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:06.174234 20664 raft_consensus.cc:2781] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:06.174405 20664 raft_consensus.cc:482] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:06.174530 20664 raft_consensus.cc:3037] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.178260 20664 raft_consensus.cc:504] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.178996 20664 leader_election.cc:288] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 election: Requested vote from peers a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745), b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783)
I20260811 00:19:06.179579 20544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1e3970d3db49568ef608ced31a6112" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50d901bb3354af58d42f9eadb9b2113"
I20260811 00:19:06.179823 20544 raft_consensus.cc:3037] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.179963 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1e3970d3db49568ef608ced31a6112" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f"
I20260811 00:19:06.180157 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.171481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37609 (local address 127.16.71.130:34947)
0811 00:19:06.171689 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.171693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.172167 (+ 474us) server_negotiation.cc:407] Connection header received
0811 00:19:06.172497 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.172502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.172579 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.172847 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:06.174328 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.175253 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.177801 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.179120 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.179303 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.179394 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.179762 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.179910 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:19:06.180202 20027 raft_consensus.cc:3037] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.180404 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.171517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:33415 (local address 127.16.71.131:43745)
0811 00:19:06.171685 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.171690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.172878 (+ 1188us) server_negotiation.cc:407] Connection header received
0811 00:19:06.173161 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.173166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.173245 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.173314 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:06.174202 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.175309 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.177974 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.179226 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.179358 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.179422 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.179971 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.180110 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:19:06.181087 20667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.170917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34947 (local address 127.16.71.129:37609)
0811 00:19:06.172065 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.172085 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.172298 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.173469 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.173473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.173502 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:06.174163 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.174179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.175470 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.175475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.177614 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.177635 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.179353 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.179360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.179551 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.179595 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.180350 (+ 755us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.180525 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"mutex_wait_us":43,"thread_start_us":1173,"threads_started":1}
I20260811 00:19:06.181272 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.171909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43745 (local address 127.16.71.129:33415)
0811 00:19:06.172775 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.172794 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.172996 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.173512 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.173517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.173543 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:06.173977 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.173996 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.175552 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.175558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.177798 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.177814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.179593 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.179601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.179766 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.179798 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.180586 (+ 788us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.180767 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":31,"thread_start_us":2489,"threads_started":1}
I20260811 00:19:06.182005 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1e3970d3db49568ef608ced31a6112" candidate_uuid: "b700a56343fe413b9cfb535ace4de46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50d901bb3354af58d42f9eadb9b2113" is_pre_election: true
I20260811 00:19:06.182090 20285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1e3970d3db49568ef608ced31a6112" candidate_uuid: "b700a56343fe413b9cfb535ace4de46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0021d6ebb63741a2b3204dfe69b32e33" is_pre_election: true
I20260811 00:19:06.182605 20285 raft_consensus.cc:2379] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b700a56343fe413b9cfb535ace4de46f in current term 1: Already voted for candidate 0021d6ebb63741a2b3204dfe69b32e33 in this term.
I20260811 00:19:06.183452 19899 leader_election.cc:302] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b700a56343fe413b9cfb535ace4de46f; no voters: 0021d6ebb63741a2b3204dfe69b32e33, a50d901bb3354af58d42f9eadb9b2113
I20260811 00:19:06.183679 20027 raft_consensus.cc:2454] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 1.
I20260811 00:19:06.183918 20661 raft_consensus.cc:2726] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 00:19:06.184177 20158 leader_election.cc:302] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0021d6ebb63741a2b3204dfe69b32e33, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:06.184491 20664 raft_consensus.cc:2781] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:06.184991 20544 raft_consensus.cc:2454] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 1.
I20260811 00:19:06.185074 20664 raft_consensus.cc:686] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Running, Role: LEADER
W20260811 00:19:06.185339 20118 tablet.cc:2369] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:06.185653 20664 consensus_queue.cc:237] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } }
I20260811 00:19:06.190742 19733 catalog_manager.cc:5495] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:19:06 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407543, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
W20260811 00:19:06.270277 20372 tablet.cc:2369] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:06.275744 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.259583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39881 (local address 127.0.0.1:51350)
0811 00:19:06.260358 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.260376 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.260533 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.260907 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.260911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.261651 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:06.261942 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.261952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.263097 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.263101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.264218 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.264232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.265100 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.265107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.265244 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.265894 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:06.265913 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:06.268173 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:06.272694 (+ 4521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:06.272710 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:06.272717 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:06.273320 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:06.273801 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:06.273805 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:06.273808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:06.274016 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:06.274692 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:06.274701 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:06.274900 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.275069 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.275335 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":443,"threads_started":1}
I20260811 00:19:06.275949 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.259432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.16.71.190:39881)
0811 00:19:06.259680 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.259686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.260476 (+ 790us) server_negotiation.cc:407] Connection header received
0811 00:19:06.260661 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.260665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.260743 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.260832 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:06.262156 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.262971 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.264348 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.264903 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.265017 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.268420 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:06.268428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:06.268433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:06.268482 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:06.272388 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:06.273475 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:06.273481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:06.273485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:06.273642 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:06.274122 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:06.274127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:06.274130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:06.274514 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:06.274659 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.275198 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.275520 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":78}
I20260811 00:19:06.281953 19733 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:51350:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:19:06.283010 19733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:06.290743 20502 tablet_service.cc:1467] Processing CreateTablet for tablet 8b86f2cb8cc7401da7716300625d6276 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=231b211cc46842719fa0c686a9a74f02]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:19:06.290575 19980 tablet_service.cc:1467] Processing CreateTablet for tablet 8b86f2cb8cc7401da7716300625d6276 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=231b211cc46842719fa0c686a9a74f02]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:19:06.290683 20247 tablet_service.cc:1467] Processing CreateTablet for tablet 8b86f2cb8cc7401da7716300625d6276 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=231b211cc46842719fa0c686a9a74f02]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:19:06.291676 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b86f2cb8cc7401da7716300625d6276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:06.291688 20502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b86f2cb8cc7401da7716300625d6276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:06.291688 20247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b86f2cb8cc7401da7716300625d6276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:06.297922 20655 tablet_bootstrap.cc:492] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: Bootstrap starting.
I20260811 00:19:06.299072 20653 tablet_bootstrap.cc:492] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: Bootstrap starting.
I20260811 00:19:06.299126 20654 tablet_bootstrap.cc:492] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: Bootstrap starting.
I20260811 00:19:06.299484 20655 tablet_bootstrap.cc:654] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.300802 20654 tablet_bootstrap.cc:654] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.300818 20653 tablet_bootstrap.cc:654] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.302634 20655 tablet_bootstrap.cc:492] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: No bootstrap required, opened a new log
I20260811 00:19:06.302786 20655 ts_tablet_manager.cc:1397] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260811 00:19:06.303260 20653 tablet_bootstrap.cc:492] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: No bootstrap required, opened a new log
I20260811 00:19:06.303413 20653 ts_tablet_manager.cc:1397] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 00:19:06.303589 20655 raft_consensus.cc:348] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.303869 20655 raft_consensus.cc:374] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.303923 20654 tablet_bootstrap.cc:492] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: No bootstrap required, opened a new log
I20260811 00:19:06.303977 20655 raft_consensus.cc:729] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b700a56343fe413b9cfb535ace4de46f, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.304085 20654 ts_tablet_manager.cc:1397] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 00:19:06.304129 20653 raft_consensus.cc:348] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.304378 20653 raft_consensus.cc:374] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.304243 20655 consensus_queue.cc:260] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.304529 20653 raft_consensus.cc:729] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d901bb3354af58d42f9eadb9b2113, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.304873 20655 ts_tablet_manager.cc:1428] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 00:19:06.304791 20653 consensus_queue.cc:260] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.305387 20653 ts_tablet_manager.cc:1428] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:19:06.305305 20662 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.305754 20662 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:06.304996 20654 raft_consensus.cc:348] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.306176 20661 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.306433 20654 raft_consensus.cc:374] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.306461 20661 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:06.306535 20654 raft_consensus.cc:729] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.306893 20654 consensus_queue.cc:260] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.307521 20654 ts_tablet_manager.cc:1428] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 00:19:06.307369 20664 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.307700 20664 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:06.498852 20664 raft_consensus.cc:482] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:06.499091 20664 raft_consensus.cc:504] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.499704 20664 leader_election.cc:288] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745)
I20260811 00:19:06.500351 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b86f2cb8cc7401da7716300625d6276" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f" is_pre_election: true
I20260811 00:19:06.500355 20544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b86f2cb8cc7401da7716300625d6276" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50d901bb3354af58d42f9eadb9b2113" is_pre_election: true
I20260811 00:19:06.500605 20027 raft_consensus.cc:2454] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 0.
I20260811 00:19:06.500609 20544 raft_consensus.cc:2454] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 0.
I20260811 00:19:06.501089 20158 leader_election.cc:302] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0021d6ebb63741a2b3204dfe69b32e33, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:06.501389 20664 raft_consensus.cc:2781] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:06.501508 20664 raft_consensus.cc:482] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:06.501601 20664 raft_consensus.cc:3037] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.503875 20664 raft_consensus.cc:504] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.504488 20664 leader_election.cc:288] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 election: Requested vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745)
I20260811 00:19:06.505100 20544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b86f2cb8cc7401da7716300625d6276" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50d901bb3354af58d42f9eadb9b2113"
I20260811 00:19:06.505203 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b86f2cb8cc7401da7716300625d6276" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f"
I20260811 00:19:06.505337 20544 raft_consensus.cc:3037] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.505522 20027 raft_consensus.cc:3037] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.507555 20544 raft_consensus.cc:2454] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 1.
I20260811 00:19:06.507911 20027 raft_consensus.cc:2454] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 1.
I20260811 00:19:06.508101 20159 leader_election.cc:302] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0021d6ebb63741a2b3204dfe69b32e33, a50d901bb3354af58d42f9eadb9b2113; no voters:
I20260811 00:19:06.508440 20664 raft_consensus.cc:2781] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:06.508665 20664 raft_consensus.cc:686] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Running, Role: LEADER
I20260811 00:19:06.508996 20664 consensus_queue.cc:237] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.510749 20671 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: TxnStatusTablet state changed. Reason: New leader 0021d6ebb63741a2b3204dfe69b32e33. Latest consensus state: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.511174 20671 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:19:06.511950 20685 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:19:06.512247 20685 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:19:06.512442 19730 catalog_manager.cc:5495] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:06.550791 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.543392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34947 (local address 127.0.0.1:53068)
0811 00:19:06.543655 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.543677 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.543813 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.544597 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.544601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.544644 (+ 43us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:19:06.544927 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.544945 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.546624 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.546629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.547726 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.547742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.548798 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.548804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.548968 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.549054 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.550270 (+ 1216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.550297 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.550378 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.550527 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 00:19:06.550915 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.543503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.16.71.130:34947)
0811 00:19:06.543711 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.543717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.543743 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:19:06.544030 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.544036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.544162 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.544350 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:06.545119 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.546359 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.547904 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.548569 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.548746 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.549264 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.550132 (+ 868us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.550288 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.550433 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.550548 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:19:06.561148 20544 raft_consensus.cc:1261] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Refusing update from remote peer 0021d6ebb63741a2b3204dfe69b32e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:06.561324 20027 raft_consensus.cc:1261] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Refusing update from remote peer 0021d6ebb63741a2b3204dfe69b32e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:06.563154 20664 consensus_queue.cc:1035] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:06.563652 20671 consensus_queue.cc:1035] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:06.576683 20661 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: TxnStatusTablet state changed. Reason: New leader 0021d6ebb63741a2b3204dfe69b32e33. Latest consensus state: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.577148 20661 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:06.582612 20662 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: TxnStatusTablet state changed. Reason: New leader 0021d6ebb63741a2b3204dfe69b32e33. Latest consensus state: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.582933 20662 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:06.583190 20692 mvcc.cc:205] Tried to move back new op lower bound from 7317125310699610112 to 7317125310502899712. Current Snapshot: MvccSnapshot[applied={T|T < 7317125310699610112}]
I20260811 00:19:06.584403 20694 mvcc.cc:205] Tried to move back new op lower bound from 7317125310699610112 to 7317125310502899712. Current Snapshot: MvccSnapshot[applied={T|T < 7317125310699610112}]
I20260811 00:19:06.585260 20661 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.585572 20661 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P b700a56343fe413b9cfb535ace4de46f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:06.589797 20671 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.590165 20671 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:19:06.594202 20664 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.594580 20664 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P 0021d6ebb63741a2b3204dfe69b32e33: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:19:06.596736 20695 mvcc.cc:205] Tried to move back new op lower bound from 7317125310699610112 to 7317125310502899712. Current Snapshot: MvccSnapshot[applied={T|T < 7317125310699610112}]
I20260811 00:19:06.605950 20662 tablet_replica.cc:439] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } } }
I20260811 00:19:06.606273 20662 tablet_replica.cc:442] T 8b86f2cb8cc7401da7716300625d6276 P a50d901bb3354af58d42f9eadb9b2113: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:19:06.608539 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.600111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53076 (local address 127.16.71.130:34947)
0811 00:19:06.600320 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.600327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.601322 (+ 995us) server_negotiation.cc:407] Connection header received
0811 00:19:06.601520 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.601526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.601631 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.601750 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:06.602298 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.603643 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.605939 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.606702 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.606835 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.607638 (+ 803us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.607792 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.607859 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.608129 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.608235 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:19:06.627817 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.621868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:45385 (local address 127.16.71.130:34947)
0811 00:19:06.622041 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.622048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.622752 (+ 704us) server_negotiation.cc:407] Connection header received
0811 00:19:06.622945 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.622948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.623005 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.623088 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:06.623612 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.624414 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.625930 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.626405 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.626524 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.627010 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.627150 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.627228 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.627358 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.627436 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:19:06.627854 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.621944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34947 (local address 127.16.71.130:45385)
0811 00:19:06.622640 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.622658 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.622819 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.623110 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.623114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.623124 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:19:06.623474 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.623485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.624534 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.624542 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.625758 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.625772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.626598 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.626603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.626726 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.626792 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.627235 (+ 443us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.627258 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.627335 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.627478 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":385,"threads_started":1}
I20260811 00:19:06.640584 20544 raft_consensus.cc:1261] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Refusing update from remote peer 0021d6ebb63741a2b3204dfe69b32e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:06.640645 20027 raft_consensus.cc:1261] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Refusing update from remote peer 0021d6ebb63741a2b3204dfe69b32e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:06.641283 20703 consensus_queue.cc:1035] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:06.641634 20664 consensus_queue.cc:1035] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:06.726181 19730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:06.726480 19730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:06.730541 19730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Sending DeleteTablet for 3 replicas of tablet ff1e3970d3db49568ef608ced31a6112
I20260811 00:19:06.731853 20237 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1e3970d3db49568ef608ced31a6112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:06 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:53058
I20260811 00:19:06.731865 20502 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1e3970d3db49568ef608ced31a6112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:06 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:45582
I20260811 00:19:06.732769 19980 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1e3970d3db49568ef608ced31a6112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:06 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:47596
I20260811 00:19:06.735793 19730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:06.737946 19730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:06.747857 19979 tablet_service.cc:1467] Processing CreateTablet for tablet 05dee0720e9d47cf8282e13a8f74f1af (DEFAULT_TABLE table=test-table [id=b0d0645787294d49bdc81761272d3959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:06.748147 20247 tablet_service.cc:1467] Processing CreateTablet for tablet 05dee0720e9d47cf8282e13a8f74f1af (DEFAULT_TABLE table=test-table [id=b0d0645787294d49bdc81761272d3959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:06.748521 19979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05dee0720e9d47cf8282e13a8f74f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:06.748745 20247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05dee0720e9d47cf8282e13a8f74f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:06.756383 20727 tablet_replica.cc:330] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: stopping tablet replica
I20260811 00:19:06.756830 20501 tablet_service.cc:1467] Processing CreateTablet for tablet 05dee0720e9d47cf8282e13a8f74f1af (DEFAULT_TABLE table=test-table [id=b0d0645787294d49bdc81761272d3959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:06.756973 20727 raft_consensus.cc:2229] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:06.757455 20727 raft_consensus.cc:2258] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:06.757496 20501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05dee0720e9d47cf8282e13a8f74f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:06.759678 20727 ts_tablet_manager.cc:1905] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:06.760535 20654 tablet_bootstrap.cc:492] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: Bootstrap starting.
I20260811 00:19:06.762421 20654 tablet_bootstrap.cc:654] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.765596 20655 tablet_bootstrap.cc:492] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: Bootstrap starting.
I20260811 00:19:06.767690 20655 tablet_bootstrap.cc:654] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.768361 20727 ts_tablet_manager.cc:1918] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:06.768597 20727 log.cc:1196] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/ff1e3970d3db49568ef608ced31a6112
I20260811 00:19:06.769236 20727 ts_tablet_manager.cc:1939] T ff1e3970d3db49568ef608ced31a6112 P a50d901bb3354af58d42f9eadb9b2113: Deleting consensus metadata
I20260811 00:19:06.771878 19708 catalog_manager.cc:4841] TS a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745): tablet ff1e3970d3db49568ef608ced31a6112 (table test-table [id=60eccac28de545f99cc0fd46dc5e0467]) successfully deleted
I20260811 00:19:06.773180 20653 tablet_bootstrap.cc:492] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: Bootstrap starting.
I20260811 00:19:06.775213 20653 tablet_bootstrap.cc:654] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:06.778936 20729 tablet_replica.cc:330] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: stopping tablet replica
I20260811 00:19:06.779309 20654 tablet_bootstrap.cc:492] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: No bootstrap required, opened a new log
I20260811 00:19:06.779490 20729 raft_consensus.cc:2229] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:06.779866 20729 raft_consensus.cc:2258] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:06.780159 20654 ts_tablet_manager.cc:1397] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.001s
I20260811 00:19:06.780720 20726 tablet_replica.cc:330] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: stopping tablet replica
I20260811 00:19:06.781332 20726 raft_consensus.cc:2229] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:06.782130 20654 raft_consensus.cc:348] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.782531 20654 raft_consensus.cc:374] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.782680 20654 raft_consensus.cc:729] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.783118 20654 consensus_queue.cc:260] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.785172 20654 ts_tablet_manager.cc:1428] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 00:19:06.786556 20726 raft_consensus.cc:2258] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:06.790354 20653 tablet_bootstrap.cc:492] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: No bootstrap required, opened a new log
I20260811 00:19:06.790563 20653 ts_tablet_manager.cc:1397] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.004s
I20260811 00:19:06.791033 20726 ts_tablet_manager.cc:1905] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:06.791640 20653 raft_consensus.cc:348] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.792171 20653 raft_consensus.cc:374] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.792266 20653 raft_consensus.cc:729] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d901bb3354af58d42f9eadb9b2113, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.792567 20653 consensus_queue.cc:260] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.793267 20653 ts_tablet_manager.cc:1428] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:06.795212 20655 tablet_bootstrap.cc:492] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: No bootstrap required, opened a new log
I20260811 00:19:06.795428 20655 ts_tablet_manager.cc:1397] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.006s
I20260811 00:19:06.798745 20655 raft_consensus.cc:348] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.799101 20655 raft_consensus.cc:374] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:06.799162 20729 ts_tablet_manager.cc:1905] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:06.799209 20655 raft_consensus.cc:729] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b700a56343fe413b9cfb535ace4de46f, State: Initialized, Role: FOLLOWER
I20260811 00:19:06.799526 20655 consensus_queue.cc:260] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.800323 20655 ts_tablet_manager.cc:1428] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260811 00:19:06.806105 20729 ts_tablet_manager.cc:1918] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:06.806317 20729 log.cc:1196] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/ff1e3970d3db49568ef608ced31a6112
I20260811 00:19:06.806915 20729 ts_tablet_manager.cc:1939] T ff1e3970d3db49568ef608ced31a6112 P b700a56343fe413b9cfb535ace4de46f: Deleting consensus metadata
I20260811 00:19:06.808763 19706 catalog_manager.cc:4841] TS b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783): tablet ff1e3970d3db49568ef608ced31a6112 (table test-table [id=60eccac28de545f99cc0fd46dc5e0467]) successfully deleted
I20260811 00:19:06.809108 20726 ts_tablet_manager.cc:1918] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:06.809320 20726 log.cc:1196] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/ff1e3970d3db49568ef608ced31a6112
I20260811 00:19:06.809882 20726 ts_tablet_manager.cc:1939] T ff1e3970d3db49568ef608ced31a6112 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting consensus metadata
I20260811 00:19:06.811991 19709 catalog_manager.cc:4841] TS 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947): tablet ff1e3970d3db49568ef608ced31a6112 (table test-table [id=60eccac28de545f99cc0fd46dc5e0467]) successfully deleted
I20260811 00:19:06.882283 20662 raft_consensus.cc:482] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:06.882526 20662 raft_consensus.cc:504] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.884364 20662 leader_election.cc:288] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947)
I20260811 00:19:06.892688 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.884751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41783 (local address 127.16.71.131:51909)
0811 00:19:06.886177 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.886202 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.886467 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.886908 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.886912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.886938 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:06.887311 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.887324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.888796 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.888802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.890484 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.890502 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.891774 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.891780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.891969 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.892006 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.892116 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.892275 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"thread_start_us":1109,"threads_started":1}
I20260811 00:19:06.892709 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.884927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:51909 (local address 127.16.71.129:41783)
0811 00:19:06.885700 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.885707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.886319 (+ 612us) server_negotiation.cc:407] Connection header received
0811 00:19:06.886659 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.886665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.886751 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.886894 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:06.887504 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.888582 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.890692 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.891554 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.891732 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.891919 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.892159 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.892313 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":798,"threads_started":1}
I20260811 00:19:06.894026 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.884887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:59285 (local address 127.16.71.130:34947)
0811 00:19:06.885220 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.885226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.886623 (+ 1397us) server_negotiation.cc:407] Connection header received
0811 00:19:06.886861 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.886867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.886961 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.887054 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:06.887861 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.888998 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.891596 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.892999 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.893206 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.893282 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.893607 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.893752 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260811 00:19:06.894045 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.885478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34947 (local address 127.16.71.131:59285)
0811 00:19:06.886533 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.886550 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.886706 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.887182 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.887188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.887218 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:06.887676 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.887691 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.889160 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.889167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.891382 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.891403 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.893272 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.893278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.893437 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.893465 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.893554 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.893700 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":754,"threads_started":1}
I20260811 00:19:06.893872 20025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05dee0720e9d47cf8282e13a8f74f1af" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f" is_pre_election: true
I20260811 00:19:06.894239 20025 raft_consensus.cc:2454] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 0.
I20260811 00:19:06.894791 20285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05dee0720e9d47cf8282e13a8f74f1af" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0021d6ebb63741a2b3204dfe69b32e33" is_pre_election: true
I20260811 00:19:06.895098 20285 raft_consensus.cc:2454] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 0.
I20260811 00:19:06.895273 20408 leader_election.cc:302] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50d901bb3354af58d42f9eadb9b2113, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:06.895756 20662 raft_consensus.cc:2781] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:06.895893 20662 raft_consensus.cc:482] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:06.896034 20662 raft_consensus.cc:3037] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.898274 20662 raft_consensus.cc:504] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.898891 20662 leader_election.cc:288] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 election: Requested vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947)
I20260811 00:19:06.899528 20025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05dee0720e9d47cf8282e13a8f74f1af" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f"
I20260811 00:19:06.899566 20285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05dee0720e9d47cf8282e13a8f74f1af" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0021d6ebb63741a2b3204dfe69b32e33"
I20260811 00:19:06.899731 20025 raft_consensus.cc:3037] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.899818 20285 raft_consensus.cc:3037] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:06.901865 20025 raft_consensus.cc:2454] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 1.
I20260811 00:19:06.902357 20408 leader_election.cc:302] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50d901bb3354af58d42f9eadb9b2113, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:06.902678 20662 raft_consensus.cc:2781] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:06.903337 20285 raft_consensus.cc:2454] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 1.
I20260811 00:19:06.903755 20662 raft_consensus.cc:686] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 1 LEADER]: Becoming Leader. State: Replica: a50d901bb3354af58d42f9eadb9b2113, State: Running, Role: LEADER
I20260811 00:19:06.904250 20662 consensus_queue.cc:237] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:06.907997 19732 catalog_manager.cc:5495] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 reported cstate change: term changed from 0 to 1, leader changed from <none> to a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131). New cstate: current_term: 1 leader_uuid: "a50d901bb3354af58d42f9eadb9b2113" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } health_report { overall_health: HEALTHY } } }
I20260811 00:19:06.950987 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.943721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.16.71.131:43745)
0811 00:19:06.944527 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.944532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.944556 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:19:06.944707 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.944712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.944814 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.945000 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:06.945629 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.946544 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.948053 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.948511 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.948673 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.949205 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.950168 (+ 963us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.950333 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.950517 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.950616 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":477,"threads_started":1}
I20260811 00:19:06.972393 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.964831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34947 (local address 127.16.71.131:53403)
0811 00:19:06.965689 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:06.965709 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:06.965951 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:06.966426 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:06.966430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:06.966441 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:19:06.966759 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.966768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.968374 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.968380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.969605 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:06.969623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.971019 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.971024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:06.971180 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.971254 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.971843 (+ 589us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.971868 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 00:19:06.971946 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 00:19:06.972080 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":367,"threads_started":1}
I20260811 00:19:06.972455 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:06.964846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:53403 (local address 127.16.71.130:34947)
0811 00:19:06.965247 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:06.965254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:06.965886 (+ 632us) server_negotiation.cc:407] Connection header received
0811 00:19:06.966126 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:06.966132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:06.966228 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:06.966351 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:06.966955 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.968157 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.969807 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:06.970675 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:06.970965 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:06.971481 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:06.971686 (+ 205us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:06.971797 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:06.972003 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 00:19:06.972104 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:19:06.983565 20025 raft_consensus.cc:1261] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Refusing update from remote peer a50d901bb3354af58d42f9eadb9b2113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:06.983704 20285 raft_consensus.cc:1261] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Refusing update from remote peer a50d901bb3354af58d42f9eadb9b2113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:06.984534 20741 consensus_queue.cc:1035] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:06.985087 20662 consensus_queue.cc:1035] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:07.030050 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:07.022673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43745 (local address 127.16.71.130:58961)
0811 00:19:07.022968 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:07.022989 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:07.023129 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:07.023526 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:07.023531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:07.023544 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:19:07.023924 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:07.023949 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:07.025515 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:07.025521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:07.027186 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:07.027210 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:07.028345 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:07.028354 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:07.028524 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:07.028641 (+ 117us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:19:07.029356 (+ 715us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:19:07.029370 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 00:19:07.029498 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 00:19:07.029676 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 00:19:07.030819 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:07.022642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:58961 (local address 127.16.71.131:43745)
0811 00:19:07.022833 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:07.022840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:07.023247 (+ 407us) server_negotiation.cc:407] Connection header received
0811 00:19:07.023319 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:07.023322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:07.023392 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:07.023468 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:07.024160 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:07.025323 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:07.027432 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:07.028104 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:07.028277 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:07.028970 (+ 693us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:07.029169 (+ 199us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:07.029284 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:07.030354 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0811 00:19:07.030478 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 00:19:07.032830 20695 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 00:19:08.649732 19732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:08.650012 19732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:08.654717 19732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Sending DeleteTablet for 3 replicas of tablet 05dee0720e9d47cf8282e13a8f74f1af
I20260811 00:19:08.655691 20237 tablet_service.cc:1514] Processing DeleteTablet for tablet 05dee0720e9d47cf8282e13a8f74f1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:08 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:53058
I20260811 00:19:08.655702 19980 tablet_service.cc:1514] Processing DeleteTablet for tablet 05dee0720e9d47cf8282e13a8f74f1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:08 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:47596
I20260811 00:19:08.655786 20501 tablet_service.cc:1514] Processing DeleteTablet for tablet 05dee0720e9d47cf8282e13a8f74f1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:08 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:45582
I20260811 00:19:08.656729 20812 tablet_replica.cc:330] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: stopping tablet replica
I20260811 00:19:08.657110 20812 raft_consensus.cc:2229] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:08.657553 20812 raft_consensus.cc:2258] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:08.659008 20812 ts_tablet_manager.cc:1905] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:08.668503 20812 ts_tablet_manager.cc:1918] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:19:08.668841 20812 log.cc:1196] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/05dee0720e9d47cf8282e13a8f74f1af
I20260811 00:19:08.669448 20812 ts_tablet_manager.cc:1939] T 05dee0720e9d47cf8282e13a8f74f1af P 0021d6ebb63741a2b3204dfe69b32e33: Deleting consensus metadata
I20260811 00:19:08.671049 19709 catalog_manager.cc:4841] TS 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947): tablet 05dee0720e9d47cf8282e13a8f74f1af (table test-table [id=b0d0645787294d49bdc81761272d3959]) successfully deleted
I20260811 00:19:08.671509 20807 tablet_replica.cc:330] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: stopping tablet replica
I20260811 00:19:08.671748 20807 raft_consensus.cc:2229] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:08.672029 20807 raft_consensus.cc:2258] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:08.672811 20819 tablet_replica.cc:330] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: stopping tablet replica
I20260811 00:19:08.673032 20819 raft_consensus.cc:2229] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:08.673549 20807 ts_tablet_manager.cc:1905] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:08.673653 20819 raft_consensus.cc:2258] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:08.674640 20819 ts_tablet_manager.cc:1905] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:08.679679 20807 ts_tablet_manager.cc:1918] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:19:08.679822 20807 log.cc:1196] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/05dee0720e9d47cf8282e13a8f74f1af
I20260811 00:19:08.680225 20807 ts_tablet_manager.cc:1939] T 05dee0720e9d47cf8282e13a8f74f1af P b700a56343fe413b9cfb535ace4de46f: Deleting consensus metadata
I20260811 00:19:08.681648 20819 ts_tablet_manager.cc:1918] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:19:08.681834 20819 log.cc:1196] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/05dee0720e9d47cf8282e13a8f74f1af
I20260811 00:19:08.681980 19706 catalog_manager.cc:4841] TS b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783): tablet 05dee0720e9d47cf8282e13a8f74f1af (table test-table [id=b0d0645787294d49bdc81761272d3959]) successfully deleted
I20260811 00:19:08.682371 20819 ts_tablet_manager.cc:1939] T 05dee0720e9d47cf8282e13a8f74f1af P a50d901bb3354af58d42f9eadb9b2113: Deleting consensus metadata
I20260811 00:19:08.683595 19708 catalog_manager.cc:4841] TS a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745): tablet 05dee0720e9d47cf8282e13a8f74f1af (table test-table [id=b0d0645787294d49bdc81761272d3959]) successfully deleted
Aug 11 00:19:08 dist-test-slave-f7th krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407548, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:19:08 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407548, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:08.729565 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:08.714509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.16.71.190:39881)
0811 00:19:08.715131 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:08.715136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:08.715159 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:19:08.715277 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:08.715282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:08.715378 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:08.715478 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:08.717013 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:08.717856 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:08.719202 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:08.719671 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:08.719805 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:08.724193 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:08.724199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:08.724203 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:08.724243 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:08.726866 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:08.727565 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:08.727570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:08.727572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:08.727705 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:08.728082 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:08.728085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:08.728087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:08.728375 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:08.728697 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:08.728949 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 00:19:08.729187 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":325,"threads_started":1}
I20260811 00:19:08.733531 19732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:08.734158 19732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:08.741268 19980 tablet_service.cc:1467] Processing CreateTablet for tablet 0f1ff8f5113d42868f2e69642d142d59 (DEFAULT_TABLE table=test-table [id=b600e938c50e491daf695d8b56650170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:08.741384 20237 tablet_service.cc:1467] Processing CreateTablet for tablet 0f1ff8f5113d42868f2e69642d142d59 (DEFAULT_TABLE table=test-table [id=b600e938c50e491daf695d8b56650170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:08.741780 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f1ff8f5113d42868f2e69642d142d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:08.741854 20237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f1ff8f5113d42868f2e69642d142d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:08.741766 20501 tablet_service.cc:1467] Processing CreateTablet for tablet 0f1ff8f5113d42868f2e69642d142d59 (DEFAULT_TABLE table=test-table [id=b600e938c50e491daf695d8b56650170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:08.742202 20501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f1ff8f5113d42868f2e69642d142d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:08.749270 20834 tablet_bootstrap.cc:492] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: Bootstrap starting.
I20260811 00:19:08.749787 20833 tablet_bootstrap.cc:492] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: Bootstrap starting.
I20260811 00:19:08.751202 20834 tablet_bootstrap.cc:654] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:08.751847 20833 tablet_bootstrap.cc:654] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:08.754781 20833 tablet_bootstrap.cc:492] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: No bootstrap required, opened a new log
I20260811 00:19:08.755090 20833 ts_tablet_manager.cc:1397] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 00:19:08.756026 20833 raft_consensus.cc:348] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.756335 20833 raft_consensus.cc:374] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:08.756457 20833 raft_consensus.cc:729] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b700a56343fe413b9cfb535ace4de46f, State: Initialized, Role: FOLLOWER
I20260811 00:19:08.756767 20833 consensus_queue.cc:260] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.757169 20836 tablet_bootstrap.cc:492] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: Bootstrap starting.
I20260811 00:19:08.757769 20833 ts_tablet_manager.cc:1428] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:19:08.760051 20836 tablet_bootstrap.cc:654] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:08.762413 20834 tablet_bootstrap.cc:492] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: No bootstrap required, opened a new log
I20260811 00:19:08.762488 20836 tablet_bootstrap.cc:492] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: No bootstrap required, opened a new log
I20260811 00:19:08.762612 20834 ts_tablet_manager.cc:1397] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260811 00:19:08.762647 20836 ts_tablet_manager.cc:1397] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 00:19:08.763466 20834 raft_consensus.cc:348] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.763677 20834 raft_consensus.cc:374] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:08.763551 20836 raft_consensus.cc:348] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.763789 20834 raft_consensus.cc:729] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Initialized, Role: FOLLOWER
I20260811 00:19:08.763824 20836 raft_consensus.cc:374] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:08.763931 20836 raft_consensus.cc:729] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d901bb3354af58d42f9eadb9b2113, State: Initialized, Role: FOLLOWER
I20260811 00:19:08.764076 20834 consensus_queue.cc:260] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.764261 20836 consensus_queue.cc:260] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.764762 20834 ts_tablet_manager.cc:1428] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 00:19:08.764945 20836 ts_tablet_manager.cc:1428] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 00:19:08.875077 20792 raft_consensus.cc:482] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:08.875383 20792 raft_consensus.cc:504] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.876132 20792 leader_election.cc:288] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947)
I20260811 00:19:08.876724 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f1ff8f5113d42868f2e69642d142d59" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f" is_pre_election: true
I20260811 00:19:08.876806 20282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f1ff8f5113d42868f2e69642d142d59" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0021d6ebb63741a2b3204dfe69b32e33" is_pre_election: true
I20260811 00:19:08.876983 20027 raft_consensus.cc:2454] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 0.
I20260811 00:19:08.877094 20282 raft_consensus.cc:2454] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 0.
I20260811 00:19:08.877549 20408 leader_election.cc:302] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50d901bb3354af58d42f9eadb9b2113, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:08.877856 20792 raft_consensus.cc:2781] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:08.877983 20792 raft_consensus.cc:482] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:08.878079 20792 raft_consensus.cc:3037] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:08.880314 20792 raft_consensus.cc:504] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.881027 20792 leader_election.cc:288] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 election: Requested vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947)
I20260811 00:19:08.881621 20282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f1ff8f5113d42868f2e69642d142d59" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0021d6ebb63741a2b3204dfe69b32e33"
I20260811 00:19:08.881639 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f1ff8f5113d42868f2e69642d142d59" candidate_uuid: "a50d901bb3354af58d42f9eadb9b2113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f"
I20260811 00:19:08.881839 20027 raft_consensus.cc:3037] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:08.881841 20282 raft_consensus.cc:3037] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:08.883962 20282 raft_consensus.cc:2454] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 1.
I20260811 00:19:08.883975 20027 raft_consensus.cc:2454] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50d901bb3354af58d42f9eadb9b2113 in term 1.
I20260811 00:19:08.884493 20408 leader_election.cc:302] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0021d6ebb63741a2b3204dfe69b32e33, a50d901bb3354af58d42f9eadb9b2113, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:08.884835 20792 raft_consensus.cc:2781] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:08.885036 20792 raft_consensus.cc:686] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 1 LEADER]: Becoming Leader. State: Replica: a50d901bb3354af58d42f9eadb9b2113, State: Running, Role: LEADER
I20260811 00:19:08.885319 20792 consensus_queue.cc:237] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:08.888365 19730 catalog_manager.cc:5495] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 reported cstate change: term changed from 0 to 1, leader changed from <none> to a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131). New cstate: current_term: 1 leader_uuid: "a50d901bb3354af58d42f9eadb9b2113" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } health_report { overall_health: HEALTHY } } }
I20260811 00:19:08.926088 20849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:08.920819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45598 (local address 127.16.71.131:43745)
0811 00:19:08.921409 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:08.921414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:08.921435 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:19:08.921494 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:08.921498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:08.921566 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:08.921658 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:08.922100 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:08.922910 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:08.924223 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:08.924725 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:08.924844 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:08.925309 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:08.925442 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:08.925520 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:08.925673 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 00:19:08.925760 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":315,"threads_started":1}
I20260811 00:19:08.936609 20282 raft_consensus.cc:1261] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Refusing update from remote peer a50d901bb3354af58d42f9eadb9b2113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:08.937192 20027 raft_consensus.cc:1261] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Refusing update from remote peer a50d901bb3354af58d42f9eadb9b2113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:08.937374 20797 consensus_queue.cc:1035] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:08.937842 20792 consensus_queue.cc:1035] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:08.991323 19732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:08.991591 19732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:08.995222 19732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Sending DeleteTablet for 3 replicas of tablet 0f1ff8f5113d42868f2e69642d142d59
I20260811 00:19:08.996263 19980 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f1ff8f5113d42868f2e69642d142d59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:08 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:47596
I20260811 00:19:08.996313 20501 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f1ff8f5113d42868f2e69642d142d59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:08 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:45582
I20260811 00:19:08.996655 20819 tablet_replica.cc:330] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: stopping tablet replica
I20260811 00:19:08.996568 20237 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f1ff8f5113d42868f2e69642d142d59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:08 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:53058
I20260811 00:19:08.996727 20807 tablet_replica.cc:330] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: stopping tablet replica
I20260811 00:19:08.996873 20812 tablet_replica.cc:330] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: stopping tablet replica
I20260811 00:19:08.996876 20819 raft_consensus.cc:2229] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:08.996953 20807 raft_consensus.cc:2229] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:08.997059 20812 raft_consensus.cc:2229] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:08.997205 20807 raft_consensus.cc:2258] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:08.997263 20812 raft_consensus.cc:2258] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:08.997259 20819 raft_consensus.cc:2258] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:08.997598 19732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 00:19:08.998490 20812 ts_tablet_manager.cc:1905] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:08.998478 20819 ts_tablet_manager.cc:1905] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:08.998505 20807 ts_tablet_manager.cc:1905] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 00:19:08.998518 19732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:09.005728 20812 ts_tablet_manager.cc:1918] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:09.005728 20819 ts_tablet_manager.cc:1918] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:09.006053 20812 log.cc:1196] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/0f1ff8f5113d42868f2e69642d142d59
I20260811 00:19:09.006536 20812 ts_tablet_manager.cc:1939] T 0f1ff8f5113d42868f2e69642d142d59 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting consensus metadata
I20260811 00:19:09.007229 20807 ts_tablet_manager.cc:1918] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:19:09.007513 20807 log.cc:1196] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/0f1ff8f5113d42868f2e69642d142d59
I20260811 00:19:09.007936 19709 catalog_manager.cc:4841] TS 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947): tablet 0f1ff8f5113d42868f2e69642d142d59 (table test-table [id=b600e938c50e491daf695d8b56650170]) successfully deleted
I20260811 00:19:09.008163 20807 ts_tablet_manager.cc:1939] T 0f1ff8f5113d42868f2e69642d142d59 P b700a56343fe413b9cfb535ace4de46f: Deleting consensus metadata
I20260811 00:19:09.008459 20237 tablet_service.cc:1467] Processing CreateTablet for tablet edb5a832319f4b47bd9196051e783eb5 (DEFAULT_TABLE table=test-table [id=45f10074dc1a42308d099da2d03fa618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:09.008965 20237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb5a832319f4b47bd9196051e783eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:09.011067 19706 catalog_manager.cc:4841] TS b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783): tablet 0f1ff8f5113d42868f2e69642d142d59 (table test-table [id=b600e938c50e491daf695d8b56650170]) successfully deleted
I20260811 00:19:09.009341 20501 tablet_service.cc:1467] Processing CreateTablet for tablet edb5a832319f4b47bd9196051e783eb5 (DEFAULT_TABLE table=test-table [id=45f10074dc1a42308d099da2d03fa618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:09.011548 19980 tablet_service.cc:1467] Processing CreateTablet for tablet edb5a832319f4b47bd9196051e783eb5 (DEFAULT_TABLE table=test-table [id=45f10074dc1a42308d099da2d03fa618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:09.011662 20501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb5a832319f4b47bd9196051e783eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:09.011996 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb5a832319f4b47bd9196051e783eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:09.005944 20819 log.cc:1196] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/0f1ff8f5113d42868f2e69642d142d59
I20260811 00:19:09.014590 20819 ts_tablet_manager.cc:1939] T 0f1ff8f5113d42868f2e69642d142d59 P a50d901bb3354af58d42f9eadb9b2113: Deleting consensus metadata
I20260811 00:19:09.016614 19708 catalog_manager.cc:4841] TS a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745): tablet 0f1ff8f5113d42868f2e69642d142d59 (table test-table [id=b600e938c50e491daf695d8b56650170]) successfully deleted
I20260811 00:19:09.017043 20834 tablet_bootstrap.cc:492] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: Bootstrap starting.
I20260811 00:19:09.018395 20836 tablet_bootstrap.cc:492] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: Bootstrap starting.
I20260811 00:19:09.019357 20834 tablet_bootstrap.cc:654] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:09.020272 20836 tablet_bootstrap.cc:654] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:09.029459 20833 tablet_bootstrap.cc:492] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: Bootstrap starting.
I20260811 00:19:09.031734 20833 tablet_bootstrap.cc:654] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:09.031947 20834 tablet_bootstrap.cc:492] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: No bootstrap required, opened a new log
I20260811 00:19:09.032192 20834 ts_tablet_manager.cc:1397] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.014s
I20260811 00:19:09.033085 20834 raft_consensus.cc:348] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.033434 20834 raft_consensus.cc:374] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:09.033576 20834 raft_consensus.cc:729] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Initialized, Role: FOLLOWER
I20260811 00:19:09.033905 20834 consensus_queue.cc:260] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.035909 20834 ts_tablet_manager.cc:1428] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 00:19:09.037175 20836 tablet_bootstrap.cc:492] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: No bootstrap required, opened a new log
I20260811 00:19:09.037590 20836 ts_tablet_manager.cc:1397] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.009s
I20260811 00:19:09.037806 20833 tablet_bootstrap.cc:492] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: No bootstrap required, opened a new log
I20260811 00:19:09.038008 20833 ts_tablet_manager.cc:1397] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 00:19:09.038810 20833 raft_consensus.cc:348] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.039141 20833 raft_consensus.cc:374] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:09.039249 20833 raft_consensus.cc:729] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b700a56343fe413b9cfb535ace4de46f, State: Initialized, Role: FOLLOWER
I20260811 00:19:09.039287 20836 raft_consensus.cc:348] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.039631 20836 raft_consensus.cc:374] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:09.039738 20836 raft_consensus.cc:729] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d901bb3354af58d42f9eadb9b2113, State: Initialized, Role: FOLLOWER
I20260811 00:19:09.039672 20833 consensus_queue.cc:260] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.040040 20836 consensus_queue.cc:260] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.041594 20833 ts_tablet_manager.cc:1428] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:19:09.042624 20836 ts_tablet_manager.cc:1428] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 00:19:09.066289 20664 raft_consensus.cc:482] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:09.066506 20664 raft_consensus.cc:504] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.067224 20664 leader_election.cc:288] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745)
I20260811 00:19:09.067895 20544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb5a832319f4b47bd9196051e783eb5" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50d901bb3354af58d42f9eadb9b2113" is_pre_election: true
I20260811 00:19:09.068128 20544 raft_consensus.cc:2454] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 0.
I20260811 00:19:09.068102 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb5a832319f4b47bd9196051e783eb5" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f" is_pre_election: true
I20260811 00:19:09.068401 20027 raft_consensus.cc:2454] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 0.
I20260811 00:19:09.068665 20159 leader_election.cc:302] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0021d6ebb63741a2b3204dfe69b32e33, a50d901bb3354af58d42f9eadb9b2113; no voters:
I20260811 00:19:09.068981 20664 raft_consensus.cc:2781] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:09.069103 20664 raft_consensus.cc:482] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:09.069201 20664 raft_consensus.cc:3037] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:09.071563 20664 raft_consensus.cc:504] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.072201 20664 leader_election.cc:288] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 election: Requested vote from peers b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783), a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745)
I20260811 00:19:09.072806 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb5a832319f4b47bd9196051e783eb5" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700a56343fe413b9cfb535ace4de46f"
I20260811 00:19:09.072846 20544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb5a832319f4b47bd9196051e783eb5" candidate_uuid: "0021d6ebb63741a2b3204dfe69b32e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50d901bb3354af58d42f9eadb9b2113"
I20260811 00:19:09.073020 20027 raft_consensus.cc:3037] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:09.073053 20544 raft_consensus.cc:3037] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:09.075136 20027 raft_consensus.cc:2454] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 1.
I20260811 00:19:09.075151 20544 raft_consensus.cc:2454] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021d6ebb63741a2b3204dfe69b32e33 in term 1.
I20260811 00:19:09.075695 20159 leader_election.cc:302] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0021d6ebb63741a2b3204dfe69b32e33, a50d901bb3354af58d42f9eadb9b2113, b700a56343fe413b9cfb535ace4de46f; no voters:
I20260811 00:19:09.076197 20664 raft_consensus.cc:2781] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:09.076345 20664 raft_consensus.cc:686] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0021d6ebb63741a2b3204dfe69b32e33, State: Running, Role: LEADER
I20260811 00:19:09.076639 20664 consensus_queue.cc:237] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } }
I20260811 00:19:09.079454 19730 catalog_manager.cc:5495] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "0021d6ebb63741a2b3204dfe69b32e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0021d6ebb63741a2b3204dfe69b32e33" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:09.101795 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:09.096106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.16.71.130:34947)
0811 00:19:09.096668 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:09.096673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:09.096692 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:19:09.096750 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:09.096754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:09.096859 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:09.096944 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:09.097312 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:09.098164 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:09.099658 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:09.100174 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:09.100278 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:09.100877 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:09.101012 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:09.101083 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:09.101312 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 00:19:09.101392 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":304,"threads_started":1}
I20260811 00:19:09.112355 20027 raft_consensus.cc:1261] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Refusing update from remote peer 0021d6ebb63741a2b3204dfe69b32e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:09.112442 20544 raft_consensus.cc:1261] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Refusing update from remote peer 0021d6ebb63741a2b3204dfe69b32e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:09.113157 20860 consensus_queue.cc:1035] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b700a56343fe413b9cfb535ace4de46f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:09.113453 20771 consensus_queue.cc:1035] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50d901bb3354af58d42f9eadb9b2113" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:19:09.135409 20758 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 00:19:10.726419 19733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:10.726647 19733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:10.731101 19733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26e2fd4ed0e041228a6454b9cda897ce: Sending DeleteTablet for 3 replicas of tablet edb5a832319f4b47bd9196051e783eb5
I20260811 00:19:10.732069 19980 tablet_service.cc:1514] Processing DeleteTablet for tablet edb5a832319f4b47bd9196051e783eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:10 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:47596
I20260811 00:19:10.732105 20237 tablet_service.cc:1514] Processing DeleteTablet for tablet edb5a832319f4b47bd9196051e783eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:10 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:53058
I20260811 00:19:10.732146 20501 tablet_service.cc:1514] Processing DeleteTablet for tablet edb5a832319f4b47bd9196051e783eb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:10 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:45582
I20260811 00:19:10.734099 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19859
I20260811 00:19:10.734515 20891 tablet_replica.cc:330] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: stopping tablet replica
I20260811 00:19:10.734776 20896 tablet_replica.cc:330] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: stopping tablet replica
I20260811 00:19:10.734805 20891 raft_consensus.cc:2229] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:10.735013 20895 tablet_replica.cc:330] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: stopping tablet replica
I20260811 00:19:10.735152 20891 raft_consensus.cc:2258] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:10.735136 20896 raft_consensus.cc:2229] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:10.735251 20895 raft_consensus.cc:2229] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:10.735527 20896 raft_consensus.cc:2258] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:10.735677 20895 raft_consensus.cc:2258] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:10.736657 20891 ts_tablet_manager.cc:1905] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:10.736856 20895 ts_tablet_manager.cc:1905] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:10.742830 20896 ts_tablet_manager.cc:1905] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:10.744726 20895 ts_tablet_manager.cc:1918] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:19:10.744974 20895 log.cc:1196] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/edb5a832319f4b47bd9196051e783eb5
I20260811 00:19:10.745383 20891 ts_tablet_manager.cc:1918] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:19:10.745486 20895 ts_tablet_manager.cc:1939] T edb5a832319f4b47bd9196051e783eb5 P 0021d6ebb63741a2b3204dfe69b32e33: Deleting consensus metadata
I20260811 00:19:10.745616 20891 log.cc:1196] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/edb5a832319f4b47bd9196051e783eb5
I20260811 00:19:10.746157 20891 ts_tablet_manager.cc:1939] T edb5a832319f4b47bd9196051e783eb5 P b700a56343fe413b9cfb535ace4de46f: Deleting consensus metadata
I20260811 00:19:10.746915 19709 catalog_manager.cc:4841] TS 0021d6ebb63741a2b3204dfe69b32e33 (127.16.71.130:34947): tablet edb5a832319f4b47bd9196051e783eb5 (table test-table [id=45f10074dc1a42308d099da2d03fa618]) successfully deleted
I20260811 00:19:10.747614 19706 catalog_manager.cc:4841] TS b700a56343fe413b9cfb535ace4de46f (127.16.71.129:41783): tablet edb5a832319f4b47bd9196051e783eb5 (table test-table [id=45f10074dc1a42308d099da2d03fa618]) successfully deleted
I20260811 00:19:10.750102 20896 ts_tablet_manager.cc:1918] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:19:10.750242 20896 log.cc:1196] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/edb5a832319f4b47bd9196051e783eb5
I20260811 00:19:10.750617 20896 ts_tablet_manager.cc:1939] T edb5a832319f4b47bd9196051e783eb5 P a50d901bb3354af58d42f9eadb9b2113: Deleting consensus metadata
I20260811 00:19:10.751782 19708 catalog_manager.cc:4841] TS a50d901bb3354af58d42f9eadb9b2113 (127.16.71.131:43745): tablet edb5a832319f4b47bd9196051e783eb5 (table test-table [id=45f10074dc1a42308d099da2d03fa618]) successfully deleted
Aug 11 00:19:10 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407548, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:10.768900 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:10.735928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55486 (local address 127.16.71.129:41783)
0811 00:19:10.748822 (+ 12894us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:10.748835 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:10.748850 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:19:10.748898 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:10.748902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:10.748962 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:10.749066 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:10.750533 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:10.751323 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:10.752837 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:10.753350 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:10.753476 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:10.762412 (+ 8936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:10.762433 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:10.762452 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:10.762506 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:10.765746 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:10.766589 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:10.766598 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:10.766601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:10.766775 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:10.767206 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:10.767211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:10.767213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:10.767618 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:10.767922 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:10.768253 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 00:19:10.768493 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12772,"thread_start_us":12596,"threads_started":1}
W20260811 00:19:10.769657 20105 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55486
W20260811 00:19:10.770380 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19859: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:10.770627 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19859
I20260811 00:19:10.815362 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20122
Aug 11 00:19:10 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407548, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:19:10.834017 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:10.816195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35106 (local address 127.16.71.130:34947)
0811 00:19:10.816872 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:10.816878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:10.816896 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:19:10.816967 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:10.816971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:10.817046 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:10.817164 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:10.818654 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:10.819850 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:10.821405 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:10.821864 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:10.821979 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:10.826870 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:10.826892 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:10.826912 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:10.826970 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:10.830526 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:10.831472 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:10.831481 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:10.831483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:10.831673 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:10.832187 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:10.832191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:10.832193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:10.832754 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:10.833026 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:10.833413 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 00:19:10.833654 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":366,"threads_started":1}
W20260811 00:19:10.834722 20361 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35106
W20260811 00:19:10.835417 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20122: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:10.835583 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 20122
I20260811 00:19:10.876406 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20378
Aug 11 00:19:10 dist-test-slave-f7th krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407548, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:19:10.894774 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:10.877307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54104 (local address 127.16.71.131:43745)
0811 00:19:10.877813 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:10.877817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:10.877836 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:19:10.877894 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:10.877897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:10.877973 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:10.878074 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:10.879510 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:10.880353 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:10.881995 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:10.882485 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:10.882611 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:10.887668 (+ 5057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:10.887689 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:10.887708 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:10.887755 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:10.891376 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:10.892273 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:10.892281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:10.892283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:10.892457 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:10.892977 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:10.892982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:10.892985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:10.893482 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:10.893796 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:10.894168 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 00:19:10.894387 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":251,"threads_started":1}
W20260811 00:19:10.895473 20610 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54104
W20260811 00:19:10.896131 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20378: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:10.896313 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 20378
I20260811 00:19:10.938323 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19675
I20260811 00:19:10.939998 19805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:11.059937 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19675
2026-08-11T00:19:11Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9069 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:11 dist-test-slave-f7th krb5kdc[20911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:11 dist-test-slave-f7th krb5kdc[20911](info): set up 2 sockets
Aug 11 00:19:11 dist-test-slave-f7th krb5kdc[20911](info): commencing operation
krb5kdc: starting...
W20260811 00:19:12.357756 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:12 dist-test-slave-f7th krb5kdc[20911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:19:12.617734 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34827
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:34827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:12.977295 20936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:12.977636 20936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:12.977728 20936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:12.985340 20936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:12.985487 20936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:12.985538 20936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:12.985608 20936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:12.985654 20936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:12.996776 20936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:34827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:12.998289 20936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:12.999761 20936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:13.014938 20944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:13.015143 20936 server_base.cc:1034] running on GCE node
W20260811 00:19:13.014876 20946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:13.014885 20948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:13.016547 20936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:13.018435 20936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:13.019841 20936 hybrid_clock.cc:648] HybridClock initialized: now 1786407553019786 us; error 40 us; skew 500 ppm
I20260811 00:19:13.020316 20936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:13 dist-test-slave-f7th krb5kdc[20911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407553, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:13.024194 20936 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:19:13.026336 20936 webserver.cc:460] Webserver started at http://127.16.71.190:42341/ using document root <none> and password file <none>
I20260811 00:19:13.027251 20936 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:13.027401 20936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:13.027765 20936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:13.031247 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "6a899eca2a9c4ab2a866f99ec62632ae"
format_stamp: "Formatted at 2026-08-11 00:19:13 on dist-test-slave-f7th"
server_key: "eb19c0c4a78051b67687ee5b0fe450f9"
server_key_iv: "9681965c2cf36f57b4133b8888e3c4c1"
server_key_version: "encryptionkey@0"
I20260811 00:19:13.032277 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "6a899eca2a9c4ab2a866f99ec62632ae"
format_stamp: "Formatted at 2026-08-11 00:19:13 on dist-test-slave-f7th"
server_key: "eb19c0c4a78051b67687ee5b0fe450f9"
server_key_iv: "9681965c2cf36f57b4133b8888e3c4c1"
server_key_version: "encryptionkey@0"
I20260811 00:19:13.039232 20936 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:19:13.043936 20959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:13.045996 20936 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 00:19:13.046264 20936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "6a899eca2a9c4ab2a866f99ec62632ae"
format_stamp: "Formatted at 2026-08-11 00:19:13 on dist-test-slave-f7th"
server_key: "eb19c0c4a78051b67687ee5b0fe450f9"
server_key_iv: "9681965c2cf36f57b4133b8888e3c4c1"
server_key_version: "encryptionkey@0"
I20260811 00:19:13.046505 20936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:13.073969 20936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:13.080683 20936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:13.081034 20936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:13.103075 20936 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:34827
I20260811 00:19:13.103098 21049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:34827 every 8 connection(s)
I20260811 00:19:13.104717 20936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:13.109335 21051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:13.113713 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 20936
I20260811 00:19:13.114012 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:13.114643 16670 external_mini_cluster.cc:1428] Setting key c133eaee8daa7b9c5cadc47125ce7ad3
I20260811 00:19:13.123291 21051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae: Bootstrap starting.
I20260811 00:19:13.127533 21051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:13.129120 21051 log.cc:824] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:19:13 dist-test-slave-f7th krb5kdc[20911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:13.133895 21051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae: No bootstrap required, opened a new log
I20260811 00:19:13.142680 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:13.116987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.16.71.190:34827)
0811 00:19:13.117941 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:13.117971 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:13.118029 (+ 58us) server_negotiation.cc:407] Connection header received
0811 00:19:13.121160 (+ 3131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:13.121202 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:13.121839 (+ 637us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:13.122356 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:13.123961 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:13.126104 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:13.127688 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:13.128874 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:13.129116 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:13.134081 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:13.134096 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:13.134119 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:13.134164 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:13.137627 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:13.138557 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:13.138566 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:13.138575 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:13.138806 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:13.139249 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:13.139254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:13.139256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:13.139940 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:13.140263 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:13.140874 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 00:19:13.141423 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":385,"threads_started":1}
I20260811 00:19:13.144143 21051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34827 } }
I20260811 00:19:13.144551 21051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:13.144685 21051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a899eca2a9c4ab2a866f99ec62632ae, State: Initialized, Role: FOLLOWER
I20260811 00:19:13.145367 21051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34827 } }
I20260811 00:19:13.145623 21051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:13.145802 21051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:13.146003 21051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:13.231142 21051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34827 } }
I20260811 00:19:13.232031 21051 leader_election.cc:302] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a899eca2a9c4ab2a866f99ec62632ae; no voters:
I20260811 00:19:13.232903 21051 leader_election.cc:288] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:13.233026 21057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:13.237061 21057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [term 1 LEADER]: Becoming Leader. State: Replica: 6a899eca2a9c4ab2a866f99ec62632ae, State: Running, Role: LEADER
I20260811 00:19:13.237718 21057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34827 } }
I20260811 00:19:13.238183 21051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:13.241122 21061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a899eca2a9c4ab2a866f99ec62632ae. Latest consensus state: current_term: 1 leader_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34827 } } }
I20260811 00:19:13.241204 21058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a899eca2a9c4ab2a866f99ec62632ae" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34827 } } }
I20260811 00:19:13.241604 21058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:13.241576 21061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:13.243259 21072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:13.254020 21072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:13.268050 21072 catalog_manager.cc:1347] Generated new cluster ID: da8c16e0bd91480fa8b825c895ca99c8
I20260811 00:19:13.268536 21072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:13.290922 21072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:13.292541 21072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:13.303830 21072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a899eca2a9c4ab2a866f99ec62632ae: Generated new TSK 0
I20260811 00:19:13.304983 21072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
W20260811 00:19:13.959311 16670 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.16.71.129: real 0.648s user 0.002s sys 0.014s
I20260811 00:19:13.960300 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34827
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:14.309665 21099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:14.310000 21099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:14.310155 21099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:14.317835 21099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:14.318010 21099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:14.318173 21099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:14.328523 21099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:14.330026 21099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:14.331436 21099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:14.347316 21109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:14.347357 21106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:14.347632 21105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:14.348519 21099 server_base.cc:1034] running on GCE node
I20260811 00:19:14.349280 21099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:14.350363 21099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:14.351609 21099 hybrid_clock.cc:648] HybridClock initialized: now 1786407554351553 us; error 43 us; skew 500 ppm
I20260811 00:19:14.352030 21099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:14 dist-test-slave-f7th krb5kdc[20911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407554, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:14.355608 21099 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:19:14.357468 21099 webserver.cc:460] Webserver started at http://127.16.71.129:43665/ using document root <none> and password file <none>
I20260811 00:19:14.358234 21099 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:14.358369 21099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:14.358721 21099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:14.362144 21099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "bc533bbe83b44b77a5f3b31c0c27bc67"
format_stamp: "Formatted at 2026-08-11 00:19:14 on dist-test-slave-f7th"
server_key: "f389f76fc3bde465ec524f97bb612860"
server_key_iv: "75e07c4d65e8d63899582aec2437140d"
server_key_version: "encryptionkey@0"
I20260811 00:19:14.363075 21099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "bc533bbe83b44b77a5f3b31c0c27bc67"
format_stamp: "Formatted at 2026-08-11 00:19:14 on dist-test-slave-f7th"
server_key: "f389f76fc3bde465ec524f97bb612860"
server_key_iv: "75e07c4d65e8d63899582aec2437140d"
server_key_version: "encryptionkey@0"
I20260811 00:19:14.369364 21099 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:19:14.373518 21125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:14.375098 21099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:19:14.375324 21099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "bc533bbe83b44b77a5f3b31c0c27bc67"
format_stamp: "Formatted at 2026-08-11 00:19:14 on dist-test-slave-f7th"
server_key: "f389f76fc3bde465ec524f97bb612860"
server_key_iv: "75e07c4d65e8d63899582aec2437140d"
server_key_version: "encryptionkey@0"
I20260811 00:19:14.375548 21099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:14.394539 21099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:14.399351 21099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:14.399617 21099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:14.400650 21099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:14.402222 21099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:14.402302 21099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:14.402405 21099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:14.402464 21099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:14.434991 21099 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:33073
I20260811 00:19:14.435053 21334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:33073 every 8 connection(s)
I20260811 00:19:14.436424 21099 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:14.445230 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21099
I20260811 00:19:14.445508 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:14.446513 16670 external_mini_cluster.cc:1428] Setting key d9a3dd45e997ce4fc67865bd914b024a
Aug 11 00:19:14 dist-test-slave-f7th krb5kdc[20911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407554, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:14.461352 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:14.439652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:39845 (local address 127.16.71.190:34827)
0811 00:19:14.440164 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:14.440169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:14.441429 (+ 1260us) server_negotiation.cc:407] Connection header received
0811 00:19:14.443489 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:14.443494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:14.443589 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:14.443693 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:14.445991 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.447020 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.448860 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.449284 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.449406 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:14.454002 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:14.454010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:14.454015 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:14.454065 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:14.457325 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:14.458587 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:14.458593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:14.458596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:14.458759 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:14.459286 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:14.459290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:14.459293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:14.459624 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:14.459807 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:14.460674 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 00:19:14.460963 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":267,"threads_started":1}
I20260811 00:19:14.462482 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:14.440307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34827 (local address 127.16.71.129:39845)
0811 00:19:14.441239 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:14.441284 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:14.443123 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:14.444023 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:14.444033 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:14.444720 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:14.445689 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:14.445722 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.447246 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.447251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:14.448677 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:14.448695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.449593 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.449603 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:14.449936 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:14.450778 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:14.450819 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:14.453656 (+ 2837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:14.457543 (+ 3887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:14.457557 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:14.457580 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:14.458338 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:14.458885 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:14.458889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:14.458892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:14.459136 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:14.459790 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:14.459798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:14.460206 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 00:19:14.460494 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 00:19:14.461129 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":375,"threads_started":1}
I20260811 00:19:14.464442 21336 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34827
I20260811 00:19:14.464885 21336 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:14.465904 21336 heartbeater.cc:511] Master 127.16.71.190:34827 requested a full tablet report, sending...
I20260811 00:19:14.468925 20998 ts_manager.cc:194] Registered new tserver with Master: bc533bbe83b44b77a5f3b31c0c27bc67 (127.16.71.129:33073)
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
I20260811 00:19:14.471134 20998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:39845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:19:14.500478 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34827
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:14.852313 21350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:14.852694 21350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:14.852861 21350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:14.860608 21350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:14.860798 21350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:14.860972 21350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:14.871418 21350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:14.873076 21350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:14.874478 21350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:14.888393 21362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:14.888356 21365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:14.888459 21363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:14.889228 21350 server_base.cc:1034] running on GCE node
I20260811 00:19:14.889947 21350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:14.890997 21350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:14.892218 21350 hybrid_clock.cc:648] HybridClock initialized: now 1786407554892156 us; error 53 us; skew 500 ppm
I20260811 00:19:14.892608 21350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:14 dist-test-slave-f7th krb5kdc[20911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407554, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:14.896046 21350 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:19:14.897744 21350 webserver.cc:460] Webserver started at http://127.16.71.130:35071/ using document root <none> and password file <none>
I20260811 00:19:14.898519 21350 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:14.898651 21350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:14.898983 21350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:14.902197 21350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "bfcad73a6d784641ab282ff8ee4c50dd"
format_stamp: "Formatted at 2026-08-11 00:19:14 on dist-test-slave-f7th"
server_key: "b7c1e80a8b0d62fe151b872cc3c3caef"
server_key_iv: "60bb01421bcf142241cb9bf85e811818"
server_key_version: "encryptionkey@0"
I20260811 00:19:14.903085 21350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "bfcad73a6d784641ab282ff8ee4c50dd"
format_stamp: "Formatted at 2026-08-11 00:19:14 on dist-test-slave-f7th"
server_key: "b7c1e80a8b0d62fe151b872cc3c3caef"
server_key_iv: "60bb01421bcf142241cb9bf85e811818"
server_key_version: "encryptionkey@0"
I20260811 00:19:14.909060 21350 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:19:14.913290 21379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:14.914956 21350 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:19:14.915154 21350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "bfcad73a6d784641ab282ff8ee4c50dd"
format_stamp: "Formatted at 2026-08-11 00:19:14 on dist-test-slave-f7th"
server_key: "b7c1e80a8b0d62fe151b872cc3c3caef"
server_key_iv: "60bb01421bcf142241cb9bf85e811818"
server_key_version: "encryptionkey@0"
I20260811 00:19:14.915387 21350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:14.934872 21350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:14.939911 21350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:14.940202 21350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:14.941684 21350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:14.943331 21350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:14.943423 21350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:14.943542 21350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:14.943614 21350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:14.977416 21350 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:37431
I20260811 00:19:14.977483 21590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:37431 every 8 connection(s)
I20260811 00:19:14.978808 21350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:14.984808 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21350
I20260811 00:19:14.984983 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:14.985424 16670 external_mini_cluster.cc:1428] Setting key 9debc220a12748d43f31ad06e9e9e0c5
Aug 11 00:19:14 dist-test-slave-f7th krb5kdc[20911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407554, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:15.001200 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:14.981742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:59223 (local address 127.16.71.190:34827)
0811 00:19:14.982256 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:14.982263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:14.983380 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 00:19:14.985822 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:14.985828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:14.985942 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:14.986045 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:14.987828 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.988654 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.990386 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.990843 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.990966 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:14.994914 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:14.994920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:14.994924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:14.994964 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:14.997675 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:14.998571 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:14.998577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:14.998579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:14.998743 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:14.999231 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:14.999235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:14.999238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:14.999550 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:14.999695 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:15.000516 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 00:19:15.000831 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":236,"threads_started":1}
I20260811 00:19:15.002365 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:14.982366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34827 (local address 127.16.71.130:59223)
0811 00:19:14.983179 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:14.983235 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:14.985446 (+ 2211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:14.986349 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:14.986359 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:14.986986 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:14.987630 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:14.987650 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.988847 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.988852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:14.990207 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:14.990223 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:14.991119 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:14.991129 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:14.991366 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:14.992115 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:14.992148 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:14.994601 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:14.997872 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:14.997884 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:14.997902 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:14.998421 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:14.998873 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:14.998878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:14.998880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:14.999109 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:14.999691 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:14.999701 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:15.000109 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 00:19:15.000389 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 00:19:15.001053 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":302,"threads_started":1}
I20260811 00:19:15.004092 21591 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34827
I20260811 00:19:15.004491 21591 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:15.005645 21591 heartbeater.cc:511] Master 127.16.71.190:34827 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:19:15.007799 20998 ts_manager.cc:194] Registered new tserver with Master: bfcad73a6d784641ab282ff8ee4c50dd (127.16.71.130:37431)
I20260811 00:19:15.008810 20998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:59223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:19:15.036216 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34827
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:15.393143 21601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:15.393466 21601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:15.393590 21601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:15.401304 21601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:15.401446 21601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:15.401597 21601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:15.411993 21601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:15.413491 21601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:15.414844 21601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:15.429091 21614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:15.429256 21609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:15.429121 21612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:15.430346 21601 server_base.cc:1034] running on GCE node
I20260811 00:19:15.431130 21601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:15.432250 21601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:15.433506 21601 hybrid_clock.cc:648] HybridClock initialized: now 1786407555433439 us; error 55 us; skew 500 ppm
I20260811 00:19:15.434003 21601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:15 dist-test-slave-f7th krb5kdc[20911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:15.437556 21601 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:19:15.439373 21601 webserver.cc:460] Webserver started at http://127.16.71.131:41205/ using document root <none> and password file <none>
I20260811 00:19:15.440145 21601 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:15.440264 21601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:15.440589 21601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:15.443864 21601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "9a2258fcc53641e9aec026df18617ab8"
format_stamp: "Formatted at 2026-08-11 00:19:15 on dist-test-slave-f7th"
server_key: "3b6ec9f9ee061b06b5a67bb38b3643f5"
server_key_iv: "1bf5849e876963c082b0412a5d5b95de"
server_key_version: "encryptionkey@0"
I20260811 00:19:15.444797 21601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "9a2258fcc53641e9aec026df18617ab8"
format_stamp: "Formatted at 2026-08-11 00:19:15 on dist-test-slave-f7th"
server_key: "3b6ec9f9ee061b06b5a67bb38b3643f5"
server_key_iv: "1bf5849e876963c082b0412a5d5b95de"
server_key_version: "encryptionkey@0"
I20260811 00:19:15.451362 21601 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:19:15.455858 21627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:15.457495 21601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:15.457677 21601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "9a2258fcc53641e9aec026df18617ab8"
format_stamp: "Formatted at 2026-08-11 00:19:15 on dist-test-slave-f7th"
server_key: "3b6ec9f9ee061b06b5a67bb38b3643f5"
server_key_iv: "1bf5849e876963c082b0412a5d5b95de"
server_key_version: "encryptionkey@0"
I20260811 00:19:15.457899 21601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:15.473155 21601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:15.474964 21336 heartbeater.cc:503] Master 127.16.71.190:34827 was elected leader, sending a full tablet report...
I20260811 00:19:15.478245 21601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:15.478560 21601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:15.479795 21601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:15.481667 21601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:15.481757 21601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:15.481849 21601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:15.481900 21601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:15.515208 21601 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:43349
I20260811 00:19:15.515242 21807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:43349 every 8 connection(s)
I20260811 00:19:15.516649 21601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:15.521135 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21601
I20260811 00:19:15.521329 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:15.521903 16670 external_mini_cluster.cc:1428] Setting key 1144e3d3c42c312c9f8c5199a11c69df
Aug 11 00:19:15 dist-test-slave-f7th krb5kdc[20911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:15.540743 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:15.519522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:52479 (local address 127.16.71.190:34827)
0811 00:19:15.520036 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:15.520044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:15.521273 (+ 1229us) server_negotiation.cc:407] Connection header received
0811 00:19:15.524074 (+ 2801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:15.524078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:15.524170 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:15.524270 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:15.526354 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:15.527248 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:15.529191 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:15.529639 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:15.529757 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:15.533756 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:15.533762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:15.533766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:15.533811 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:15.536838 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:15.538017 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:15.538024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:15.538027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:15.538250 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:15.538922 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:15.538926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:15.538928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:15.539232 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:15.539389 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:15.540162 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 00:19:15.540390 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":221,"threads_started":1}
I20260811 00:19:15.541987 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:15.520068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34827 (local address 127.16.71.131:52479)
0811 00:19:15.521045 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:15.521114 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:15.523450 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:15.524716 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:15.524727 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:15.525370 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:15.526140 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:15.526164 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:15.527451 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:15.527457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:15.528996 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:15.529015 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:15.529897 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:15.529903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:15.530178 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:15.530902 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:15.530937 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:15.533406 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:15.537030 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:15.537040 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:15.537057 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:15.537809 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:15.538409 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:15.538414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:15.538418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:15.538776 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:15.539369 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:15.539377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:15.539790 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 00:19:15.540040 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 00:19:15.540519 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":338,"threads_started":1}
I20260811 00:19:15.543593 21810 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34827
I20260811 00:19:15.543977 21810 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:15.544847 21810 heartbeater.cc:511] Master 127.16.71.190:34827 requested a full tablet report, sending...
I20260811 00:19:15.546712 20998 ts_manager.cc:194] Registered new tserver with Master: 9a2258fcc53641e9aec026df18617ab8 (127.16.71.131:43349)
I20260811 00:19:15.547665 20998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:52479
I20260811 00:19:15.551659 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:19:15.594452 21814 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:15.591416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.16.71.190:34827)
0811 00:19:15.591685 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:15.591692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:15.592014 (+ 322us) server_negotiation.cc:407] Connection header received
0811 00:19:15.592199 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:15.592204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:15.592282 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:15.592378 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:15.594152 (+ 1774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54728: BlockingRecv error: recv got EOF from 127.0.0.1:54728 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 00:19:15.598002 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21099
W20260811 00:19:15.601485 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21099: Not authorized: Client connection negotiation failed: client connection to 127.16.71.129:33073: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:19:15.601701 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21099
I20260811 00:19:15.640389 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21350
W20260811 00:19:15.643384 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21350: Not authorized: Client connection negotiation failed: client connection to 127.16.71.130:37431: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:19:15.643580 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21350
I20260811 00:19:15.678622 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21601
W20260811 00:19:15.681838 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21601: Not authorized: Client connection negotiation failed: client connection to 127.16.71.131:43349: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:19:15.682049 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21601
I20260811 00:19:15.712059 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20936
I20260811 00:19:15.713402 21043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:15.820273 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 20936
2026-08-11T00:19:15Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4755 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:15 dist-test-slave-f7th krb5kdc[21843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:15 dist-test-slave-f7th krb5kdc[21843](info): set up 2 sockets
Aug 11 00:19:15 dist-test-slave-f7th krb5kdc[21843](info): commencing operation
krb5kdc: starting...
W20260811 00:19:17.114152 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:17 dist-test-slave-f7th krb5kdc[21843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:19:17.354133 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:38695
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:17.715029 21864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:17.715382 21864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:17.715489 21864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:17.723431 21864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:17.723579 21864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:17.723629 21864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:17.723712 21864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:17.723783 21864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:17.735067 21864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:38695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:38695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:17.736593 21864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:17.738178 21864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:17.751051 21877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:17.751061 21879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:17.751081 21876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:17.751914 21864 server_base.cc:1034] running on GCE node
I20260811 00:19:17.752908 21864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:17.754520 21864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:17.755795 21864 hybrid_clock.cc:648] HybridClock initialized: now 1786407557755729 us; error 58 us; skew 500 ppm
I20260811 00:19:17.756254 21864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:17 dist-test-slave-f7th krb5kdc[21843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:17.759845 21864 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:19:17.761720 21864 webserver.cc:460] Webserver started at http://127.16.71.190:45363/ using document root <none> and password file <none>
I20260811 00:19:17.762542 21864 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:17.762674 21864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:17.763026 21864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:17.766515 21864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "7c2756e2fee2477e98cf035a27ed3515"
format_stamp: "Formatted at 2026-08-11 00:19:17 on dist-test-slave-f7th"
server_key: "39a538e90a7aa380fa2df09800ff46e7"
server_key_iv: "481904b2a8c96cfa5379180bf72ecf92"
server_key_version: "encryptionkey@0"
I20260811 00:19:17.767419 21864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "7c2756e2fee2477e98cf035a27ed3515"
format_stamp: "Formatted at 2026-08-11 00:19:17 on dist-test-slave-f7th"
server_key: "39a538e90a7aa380fa2df09800ff46e7"
server_key_iv: "481904b2a8c96cfa5379180bf72ecf92"
server_key_version: "encryptionkey@0"
I20260811 00:19:17.773751 21864 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:19:17.777843 21887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:17.779515 21864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:19:17.779723 21864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "7c2756e2fee2477e98cf035a27ed3515"
format_stamp: "Formatted at 2026-08-11 00:19:17 on dist-test-slave-f7th"
server_key: "39a538e90a7aa380fa2df09800ff46e7"
server_key_iv: "481904b2a8c96cfa5379180bf72ecf92"
server_key_version: "encryptionkey@0"
I20260811 00:19:17.779944 21864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:17.809643 21864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:17.814930 21864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:17.815254 21864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:17.836190 21864 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:38695
I20260811 00:19:17.836210 21970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:38695 every 8 connection(s)
I20260811 00:19:17.837867 21864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:17.840060 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21864
I20260811 00:19:17.840481 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:17.841251 16670 external_mini_cluster.cc:1428] Setting key 138f12c3205089aad007dab22ad56ccd
I20260811 00:19:17.842579 21973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:19:17 dist-test-slave-f7th krb5kdc[21843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:17.858371 21973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515: Bootstrap starting.
I20260811 00:19:17.862141 21973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:17.863296 21973 log.cc:824] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:17.866520 21973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515: No bootstrap required, opened a new log
I20260811 00:19:17.866760 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:17.843276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.16.71.190:38695)
0811 00:19:17.844191 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:17.844203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:17.844248 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:19:17.846013 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:17.846043 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:17.846527 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:17.846938 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:17.849100 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:17.850568 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:17.852369 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:17.853210 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:17.853376 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:17.858801 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:17.858813 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:17.858831 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:17.858883 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:17.862264 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:17.863116 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:17.863129 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:17.863137 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:17.863331 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:17.863763 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:17.863767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:17.863769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:17.864407 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:17.864754 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:17.865253 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 00:19:17.865701 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":415,"threads_started":1}
I20260811 00:19:17.871574 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:19:17.875361 21973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2756e2fee2477e98cf035a27ed3515" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38695 } }
I20260811 00:19:17.875850 21973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:17.875962 21973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2756e2fee2477e98cf035a27ed3515, State: Initialized, Role: FOLLOWER
I20260811 00:19:17.876775 21973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2756e2fee2477e98cf035a27ed3515" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38695 } }
I20260811 00:19:17.877058 21973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:17.877213 21973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:17.877382 21973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:17.879694 21973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2756e2fee2477e98cf035a27ed3515" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38695 } }
I20260811 00:19:17.880290 21973 leader_election.cc:302] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c2756e2fee2477e98cf035a27ed3515; no voters:
I20260811 00:19:17.881022 21973 leader_election.cc:288] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:17.881153 21984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:17.882386 21984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [term 1 LEADER]: Becoming Leader. State: Replica: 7c2756e2fee2477e98cf035a27ed3515, State: Running, Role: LEADER
I20260811 00:19:17.883102 21984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2756e2fee2477e98cf035a27ed3515" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38695 } }
I20260811 00:19:17.883669 21973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:17.890399 21985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c2756e2fee2477e98cf035a27ed3515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2756e2fee2477e98cf035a27ed3515" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38695 } } }
I20260811 00:19:17.890935 21985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:17.891573 21987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2756e2fee2477e98cf035a27ed3515. Latest consensus state: current_term: 1 leader_uuid: "7c2756e2fee2477e98cf035a27ed3515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2756e2fee2477e98cf035a27ed3515" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38695 } } }
I20260811 00:19:17.891928 21987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:17.913264 21992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:17.921077 21992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:17.949208 21992 catalog_manager.cc:1347] Generated new cluster ID: 4aab92fbd5fc467ca694d5baf32bfcb5
I20260811 00:19:17.949363 21992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:17.949787 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:17.930947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.16.71.190:38695)
0811 00:19:17.931155 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:17.931161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:17.938469 (+ 7308us) server_negotiation.cc:407] Connection header received
0811 00:19:17.938891 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:17.938898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:17.939006 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:17.939093 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:17.941093 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:17.942308 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:17.944471 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:17.945357 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:17.945537 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:17.947237 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:17.947245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:17.947250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:17.948216 (+ 966us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:17.948312 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:17.948359 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:17.948484 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:17.949247 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 00:19:17.949366 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:19:17.958988 21992 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:17.960382 21992 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:17.970444 21992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c2756e2fee2477e98cf035a27ed3515: Generated new TSK 0
I20260811 00:19:17.971446 21992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:19:17.991314 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21864
I20260811 00:19:17.992870 21962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:18.088725 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21864
2026-08-11T00:19:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2262 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:18 dist-test-slave-f7th krb5kdc[22023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:18 dist-test-slave-f7th krb5kdc[22023](info): set up 2 sockets
Aug 11 00:19:18 dist-test-slave-f7th krb5kdc[22023](info): commencing operation
krb5kdc: starting...
W20260811 00:19:19.376214 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:19 dist-test-slave-f7th krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:19:19.621644 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39273
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:39273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:19.976075 22047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:19.976428 22047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:19.976579 22047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:19.984297 22047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:19.984447 22047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:19.984494 22047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:19.984543 22047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:19.984588 22047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:19.995723 22047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:39273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:19.997231 22047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:19.998627 22047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:20.011078 22056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:20.011108 22057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:20.011047 22060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:20.011896 22047 server_base.cc:1034] running on GCE node
I20260811 00:19:20.012809 22047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:20.014338 22047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:20.015594 22047 hybrid_clock.cc:648] HybridClock initialized: now 1786407560015531 us; error 49 us; skew 500 ppm
I20260811 00:19:20.015961 22047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:20 dist-test-slave-f7th krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407560, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:20.019447 22047 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:19:20.021189 22047 webserver.cc:460] Webserver started at http://127.16.71.190:40287/ using document root <none> and password file <none>
I20260811 00:19:20.022115 22047 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:20.022292 22047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:20.022648 22047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:20.026121 22047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "62af0cb418fb4719a1120186d9989466"
format_stamp: "Formatted at 2026-08-11 00:19:20 on dist-test-slave-f7th"
server_key: "dda32fb5e3725bfcabec845ac02e2a80"
server_key_iv: "3299432cd6f90896a34876ceba327c7a"
server_key_version: "encryptionkey@0"
I20260811 00:19:20.027040 22047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "62af0cb418fb4719a1120186d9989466"
format_stamp: "Formatted at 2026-08-11 00:19:20 on dist-test-slave-f7th"
server_key: "dda32fb5e3725bfcabec845ac02e2a80"
server_key_iv: "3299432cd6f90896a34876ceba327c7a"
server_key_version: "encryptionkey@0"
I20260811 00:19:20.033238 22047 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:19:20.037621 22077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:20.039458 22047 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:19:20.039673 22047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "62af0cb418fb4719a1120186d9989466"
format_stamp: "Formatted at 2026-08-11 00:19:20 on dist-test-slave-f7th"
server_key: "dda32fb5e3725bfcabec845ac02e2a80"
server_key_iv: "3299432cd6f90896a34876ceba327c7a"
server_key_version: "encryptionkey@0"
I20260811 00:19:20.039906 22047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:20.053817 22047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:20.058475 22047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:20.058765 22047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:20.077697 22047 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:39273
I20260811 00:19:20.077719 22157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:39273 every 8 connection(s)
I20260811 00:19:20.079205 22047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:20.083302 22158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:20.087379 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22047
I20260811 00:19:20.087622 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:20.088296 16670 external_mini_cluster.cc:1428] Setting key f789059fc95871d681c6ae70ea0400aa
I20260811 00:19:20.095517 22158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466: Bootstrap starting.
I20260811 00:19:20.099036 22158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:20.100500 22158 log.cc:824] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:20.103967 22158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466: No bootstrap required, opened a new log
Aug 11 00:19:20 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:20.114609 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:20.090682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.16.71.190:39273)
0811 00:19:20.091589 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:20.091601 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:20.091642 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:19:20.093931 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:20.093964 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:20.094451 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:20.094897 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:20.096511 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:20.097932 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:20.099265 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:20.100365 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:20.100584 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:20.105173 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:20.105186 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:20.105212 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:20.105268 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:20.109488 (+ 4220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:20.110324 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:20.110336 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:20.110346 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:20.110618 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:20.110991 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:20.110997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:20.111000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:20.111813 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:20.112157 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:20.112544 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 00:19:20.113116 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":417,"threads_started":1}
I20260811 00:19:20.114789 22158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62af0cb418fb4719a1120186d9989466" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39273 } }
I20260811 00:19:20.115267 22158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:20.115415 22158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62af0cb418fb4719a1120186d9989466, State: Initialized, Role: FOLLOWER
I20260811 00:19:20.116114 22158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62af0cb418fb4719a1120186d9989466" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39273 } }
I20260811 00:19:20.116376 22158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:20.116530 22158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:20.116740 22158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:20.119477 22158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62af0cb418fb4719a1120186d9989466" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39273 } }
I20260811 00:19:20.120122 22158 leader_election.cc:302] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62af0cb418fb4719a1120186d9989466; no voters:
I20260811 00:19:20.120893 22158 leader_election.cc:288] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:20.121021 22166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:20.121665 22166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [term 1 LEADER]: Becoming Leader. State: Replica: 62af0cb418fb4719a1120186d9989466, State: Running, Role: LEADER
I20260811 00:19:20.122335 22166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62af0cb418fb4719a1120186d9989466" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39273 } }
I20260811 00:19:20.123323 22158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:20.125341 22169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62af0cb418fb4719a1120186d9989466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62af0cb418fb4719a1120186d9989466" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39273 } } }
I20260811 00:19:20.125756 22169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:20.126327 22170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62af0cb418fb4719a1120186d9989466. Latest consensus state: current_term: 1 leader_uuid: "62af0cb418fb4719a1120186d9989466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62af0cb418fb4719a1120186d9989466" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39273 } } }
I20260811 00:19:20.126641 22170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:20.127437 22176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:20.133695 22176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:20.144083 22176 catalog_manager.cc:1347] Generated new cluster ID: 40090716eef6456c897543037dcd889a
I20260811 00:19:20.144243 22176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:20.177372 22176 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:20.179396 22176 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:20.188411 22176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62af0cb418fb4719a1120186d9989466: Generated new TSK 0
I20260811 00:19:20.189399 22176 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:19:20.257747 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39273
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:20.621532 22207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:20.621866 22207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:20.621984 22207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:20.630079 22207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:20.630224 22207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:20.630398 22207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:20.640882 22207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:20.642304 22207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:20.643731 22207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:20.657116 22218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:20.657203 22217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:20.657176 22221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:20.658265 22207 server_base.cc:1034] running on GCE node
I20260811 00:19:20.658946 22207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:20.659972 22207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:20.661224 22207 hybrid_clock.cc:648] HybridClock initialized: now 1786407560661148 us; error 63 us; skew 500 ppm
I20260811 00:19:20.661633 22207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:20 dist-test-slave-f7th krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407560, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:20.665373 22207 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:19:20.667160 22207 webserver.cc:460] Webserver started at http://127.16.71.129:34091/ using document root <none> and password file <none>
I20260811 00:19:20.667929 22207 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:20.668068 22207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:20.668381 22207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:20.671922 22207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "b77ef787913c4d999de205967f2b92f5"
format_stamp: "Formatted at 2026-08-11 00:19:20 on dist-test-slave-f7th"
server_key: "13f98c2f776babefe0fd2ee4cc969359"
server_key_iv: "7e2cf193eac3c9be5ea7a1396d7be994"
server_key_version: "encryptionkey@0"
I20260811 00:19:20.672886 22207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "b77ef787913c4d999de205967f2b92f5"
format_stamp: "Formatted at 2026-08-11 00:19:20 on dist-test-slave-f7th"
server_key: "13f98c2f776babefe0fd2ee4cc969359"
server_key_iv: "7e2cf193eac3c9be5ea7a1396d7be994"
server_key_version: "encryptionkey@0"
I20260811 00:19:20.678658 22207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:19:20.682750 22230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:20.684372 22207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:19:20.684592 22207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "b77ef787913c4d999de205967f2b92f5"
format_stamp: "Formatted at 2026-08-11 00:19:20 on dist-test-slave-f7th"
server_key: "13f98c2f776babefe0fd2ee4cc969359"
server_key_iv: "7e2cf193eac3c9be5ea7a1396d7be994"
server_key_version: "encryptionkey@0"
I20260811 00:19:20.684881 22207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:20.696908 22207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:20.701692 22207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:20.702018 22207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:20.703094 22207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:20.704635 22207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:20.704731 22207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:19:20.704861 22207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:20.704917 22207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:20.738109 22207 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:42823
I20260811 00:19:20.738149 22438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:42823 every 8 connection(s)
I20260811 00:19:20.739504 22207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:20.743810 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22207
I20260811 00:19:20.744050 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:20.744740 16670 external_mini_cluster.cc:1428] Setting key 39d3a6055d4181c5cad704cee6bcb973
Aug 11 00:19:20 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407560, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:20.762143 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:20.742179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:38967 (local address 127.16.71.190:39273)
0811 00:19:20.742671 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:20.742677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:20.743792 (+ 1115us) server_negotiation.cc:407] Connection header received
0811 00:19:20.746062 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:20.746067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:20.746154 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:20.746254 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:20.748027 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:20.748808 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:20.750446 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:20.750858 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:20.750985 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:20.755169 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:20.755176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:20.755179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:20.755222 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:20.757993 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:20.759210 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:20.759216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:20.759219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:20.759408 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:20.760003 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:20.760008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:20.760010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:20.760358 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:20.760485 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:20.761624 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0811 00:19:20.761853 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
I20260811 00:19:20.763842 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:20.742800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39273 (local address 127.16.71.129:38967)
0811 00:19:20.743565 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:20.743618 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:20.745766 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:20.746592 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:20.746604 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:20.747195 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:20.747831 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:20.747849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:20.748987 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:20.748991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:20.750271 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:20.750291 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:20.751102 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:20.751111 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:20.751349 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:20.752040 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:20.752065 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:20.754864 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:20.758220 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:20.758235 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:20.758260 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:20.759014 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:20.759551 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:20.759557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:20.759560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:20.759855 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:20.760514 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:20.760523 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:20.761041 (+ 518us) client_negotiation.cc:769] Sending connection context
0811 00:19:20.761399 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 00:19:20.762241 (+ 842us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":290,"threads_started":1}
I20260811 00:19:20.765841 22440 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39273
I20260811 00:19:20.766206 22440 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:20.767046 22440 heartbeater.cc:511] Master 127.16.71.190:39273 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
I20260811 00:19:20.769690 22104 ts_manager.cc:194] Registered new tserver with Master: b77ef787913c4d999de205967f2b92f5 (127.16.71.129:42823)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
I20260811 00:19:20.771492 22104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:38967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:19:20.800438 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39273
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:21.165665 22453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:21.165982 22453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:21.166082 22453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:21.173942 22453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:21.174090 22453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:21.174209 22453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:21.185178 22453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:21.186589 22453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:21.188028 22453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:21.202628 22462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:21.202597 22465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:21.202693 22463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:21.203876 22453 server_base.cc:1034] running on GCE node
I20260811 00:19:21.204605 22453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:21.205789 22453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:21.207026 22453 hybrid_clock.cc:648] HybridClock initialized: now 1786407561206965 us; error 52 us; skew 500 ppm
I20260811 00:19:21.207388 22453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:21 dist-test-slave-f7th krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:21.213912 22453 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:19:21.215797 22453 webserver.cc:460] Webserver started at http://127.16.71.130:33797/ using document root <none> and password file <none>
I20260811 00:19:21.216696 22453 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:21.216848 22453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:21.217173 22453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:21.220398 22453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "09f7919539de4d7fb4e45c28ab068c9b"
format_stamp: "Formatted at 2026-08-11 00:19:21 on dist-test-slave-f7th"
server_key: "ee179b5dda90a38815a04903afb42888"
server_key_iv: "a15fd3c5265ea25cc69438834e3f2c3a"
server_key_version: "encryptionkey@0"
I20260811 00:19:21.221328 22453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "09f7919539de4d7fb4e45c28ab068c9b"
format_stamp: "Formatted at 2026-08-11 00:19:21 on dist-test-slave-f7th"
server_key: "ee179b5dda90a38815a04903afb42888"
server_key_iv: "a15fd3c5265ea25cc69438834e3f2c3a"
server_key_version: "encryptionkey@0"
I20260811 00:19:21.227288 22453 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:19:21.231523 22484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:21.233217 22453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:21.233444 22453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "09f7919539de4d7fb4e45c28ab068c9b"
format_stamp: "Formatted at 2026-08-11 00:19:21 on dist-test-slave-f7th"
server_key: "ee179b5dda90a38815a04903afb42888"
server_key_iv: "a15fd3c5265ea25cc69438834e3f2c3a"
server_key_version: "encryptionkey@0"
I20260811 00:19:21.233657 22453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:21.252810 22453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:21.257923 22453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:21.258255 22453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:21.259341 22453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:21.260915 22453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:21.261015 22453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:21.261135 22453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:21.261188 22453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:21.292418 22453 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:32817
I20260811 00:19:21.292462 22669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:32817 every 8 connection(s)
I20260811 00:19:21.293848 22453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:21.297708 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22453
I20260811 00:19:21.297969 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:21.298591 16670 external_mini_cluster.cc:1428] Setting key c43db177f0ba89a23f8a6329859e02a2
Aug 11 00:19:21 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:21.317551 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:21.296778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:60293 (local address 127.16.71.190:39273)
0811 00:19:21.297352 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:21.297357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:21.298772 (+ 1415us) server_negotiation.cc:407] Connection header received
0811 00:19:21.300672 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:21.300678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:21.300768 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:21.300878 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:21.302634 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.303398 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.305206 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.305676 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.305802 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:21.310187 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:21.310196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:21.310201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:21.310247 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:21.313633 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.314629 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.314635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.314638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.314810 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.315350 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.315357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.315360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.315734 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:21.315926 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:21.316881 (+ 955us) server_negotiation.cc:299] Negotiation successful
0811 00:19:21.317155 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":334,"threads_started":1}
I20260811 00:19:21.318579 22676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:21.297541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39273 (local address 127.16.71.130:60293)
0811 00:19:21.298513 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:21.298565 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:21.300337 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:21.301174 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:21.301185 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:21.301766 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:21.302423 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:21.302445 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.303596 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.303601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:21.305023 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:21.305040 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.306049 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.306061 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:21.306330 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:21.307180 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:21.307211 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:21.309859 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:21.313859 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:21.313871 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:21.313892 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:21.314458 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:21.314955 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:21.314959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:21.314961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:21.315194 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:21.315924 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:21.315932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:21.316382 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 00:19:21.316740 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 00:19:21.317242 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":412,"threads_started":1}
I20260811 00:19:21.320531 22671 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39273
I20260811 00:19:21.320968 22671 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:21.321821 22671 heartbeater.cc:511] Master 127.16.71.190:39273 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:19:21.323683 22104 ts_manager.cc:194] Registered new tserver with Master: 09f7919539de4d7fb4e45c28ab068c9b (127.16.71.130:32817)
I20260811 00:19:21.324788 22104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:60293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:19:21.356832 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39273
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:21.726636 22679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:21.727005 22679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:21.727133 22679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:21.735113 22679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:21.735258 22679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:21.735419 22679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:21.748405 22679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:21.749873 22679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:21.751263 22679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:21.765724 22690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:21.765787 22701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:21.765906 22689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:21.766767 22679 server_base.cc:1034] running on GCE node
I20260811 00:19:21.767561 22679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:21.768774 22679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:21.770129 22679 hybrid_clock.cc:648] HybridClock initialized: now 1786407561770048 us; error 71 us; skew 500 ppm
I20260811 00:19:21.770534 22679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:21 dist-test-slave-f7th krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:21.774444 22679 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:19:21.775039 22440 heartbeater.cc:503] Master 127.16.71.190:39273 was elected leader, sending a full tablet report...
I20260811 00:19:21.776568 22679 webserver.cc:460] Webserver started at http://127.16.71.131:41155/ using document root <none> and password file <none>
I20260811 00:19:21.777599 22679 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:21.777742 22679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:21.778088 22679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:21.781778 22679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "da07468fb1fb43f3b38a9beec3e63caa"
format_stamp: "Formatted at 2026-08-11 00:19:21 on dist-test-slave-f7th"
server_key: "ae68ed9a42ab07c0825666c47eb342cd"
server_key_iv: "f80b77a9ad9ad9c77f0c62765f6297aa"
server_key_version: "encryptionkey@0"
I20260811 00:19:21.782824 22679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "da07468fb1fb43f3b38a9beec3e63caa"
format_stamp: "Formatted at 2026-08-11 00:19:21 on dist-test-slave-f7th"
server_key: "ae68ed9a42ab07c0825666c47eb342cd"
server_key_iv: "f80b77a9ad9ad9c77f0c62765f6297aa"
server_key_version: "encryptionkey@0"
I20260811 00:19:21.789101 22679 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 00:19:21.793406 22717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:21.795055 22679 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:19:21.795274 22679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "da07468fb1fb43f3b38a9beec3e63caa"
format_stamp: "Formatted at 2026-08-11 00:19:21 on dist-test-slave-f7th"
server_key: "ae68ed9a42ab07c0825666c47eb342cd"
server_key_iv: "f80b77a9ad9ad9c77f0c62765f6297aa"
server_key_version: "encryptionkey@0"
I20260811 00:19:21.795506 22679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:21.815853 22679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:21.821365 22679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:21.821722 22679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:21.822872 22679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:21.824746 22679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:21.824862 22679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:21.825004 22679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:21.825098 22679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:21.860766 22679 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:42207
I20260811 00:19:21.860846 22921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:42207 every 8 connection(s)
I20260811 00:19:21.862171 22679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:21.864188 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22679
I20260811 00:19:21.864388 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:21.865006 16670 external_mini_cluster.cc:1428] Setting key 8442c7b068812deaa87c4cee549968e7
Aug 11 00:19:21 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:21.885671 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:21.865184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:58979 (local address 127.16.71.190:39273)
0811 00:19:21.865685 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:21.865689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:21.866991 (+ 1302us) server_negotiation.cc:407] Connection header received
0811 00:19:21.869308 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:21.869313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:21.869414 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:21.869518 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:21.871292 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.872107 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.874087 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.874550 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.874672 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:21.878727 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:21.878733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:21.878736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:21.878780 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:21.882140 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.883017 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.883021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.883024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.883161 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.883739 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.883744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.883746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.884070 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:21.884214 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:21.885149 (+ 935us) server_negotiation.cc:299] Negotiation successful
0811 00:19:21.885356 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":225,"threads_started":1}
I20260811 00:19:21.886730 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:21.865915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39273 (local address 127.16.71.131:58979)
0811 00:19:21.866824 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:21.866869 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:21.868990 (+ 2121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:21.869818 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:21.869828 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:21.870423 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:21.871089 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:21.871110 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.872287 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.872291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:21.873896 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:21.873915 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.874817 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.874826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:21.875078 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:21.875777 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:21.875806 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:21.878394 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:21.882319 (+ 3925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:21.882328 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:21.882346 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:21.882876 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:21.883305 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:21.883309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:21.883312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:21.883576 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:21.884226 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:21.884234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:21.884731 (+ 497us) client_negotiation.cc:769] Sending connection context
0811 00:19:21.885025 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 00:19:21.885513 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":362,"threads_started":1}
I20260811 00:19:21.888391 22922 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39273
I20260811 00:19:21.888809 22922 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:21.889714 22922 heartbeater.cc:511] Master 127.16.71.190:39273 requested a full tablet report, sending...
I20260811 00:19:21.891534 22104 ts_manager.cc:194] Registered new tserver with Master: da07468fb1fb43f3b38a9beec3e63caa (127.16.71.131:42207)
I20260811 00:19:21.892482 22104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:58979
I20260811 00:19:21.894551 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:19:21 dist-test-slave-f7th krb5kdc[22023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 00:19:21 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:21.956373 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:21.938172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.16.71.190:39273)
0811 00:19:21.938505 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:21.938512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:21.939766 (+ 1254us) server_negotiation.cc:407] Connection header received
0811 00:19:21.940065 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:21.940070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:21.940170 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:21.940292 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:21.942127 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.942973 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.944954 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.945500 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.945635 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:21.950469 (+ 4834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:21.950476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:21.950480 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:21.950534 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:21.953556 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.954449 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.954455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.954457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.954613 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.954975 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.954978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.954980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.955364 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:21.955577 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:21.955854 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 00:19:21.956087 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
W20260811 00:19:21.957085 22104 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46046
I20260811 00:19:21.959751 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22207
Aug 11 00:19:21 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:21.976981 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:21.960900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.16.71.129:42823)
0811 00:19:21.961570 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:21.961578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:21.961610 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:19:21.961672 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:21.961678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:21.961888 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:21.962022 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:21.963271 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.964311 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.965935 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:21.966608 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:21.966727 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:21.971097 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:21.971115 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:21.971133 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:21.971193 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:21.974349 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.975102 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.975109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.975111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.975246 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:21.975583 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:21.975586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:21.975588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:21.975942 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:21.976164 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:21.976413 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 00:19:21.976664 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":316,"threads_started":1}
W20260811 00:19:21.978349 22433 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36314
W20260811 00:19:21.979480 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22207: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:21.979657 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22207
I20260811 00:19:22.006843 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22453
Aug 11 00:19:22 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:19:22.026232 22944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:22.007940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.16.71.130:32817)
0811 00:19:22.008451 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:22.008458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:22.008485 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:19:22.008559 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:22.008567 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:22.008921 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:22.009106 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:22.010875 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:22.012002 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:22.013489 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:22.014170 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:22.014309 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:22.019068 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:22.019087 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:22.019105 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:22.019150 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:22.022635 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:22.023612 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:22.023622 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:22.023625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:22.023804 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:22.024298 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:22.024305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:22.024308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:22.024794 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:22.025151 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:22.025526 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 00:19:22.025833 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
W20260811 00:19:22.027654 22665 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33194
W20260811 00:19:22.029130 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22453: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:22.029322 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22453
I20260811 00:19:22.052911 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22679
Aug 11 00:19:22 dist-test-slave-f7th krb5kdc[22023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407561, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:19:22.071473 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:22.053931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.16.71.131:42207)
0811 00:19:22.054629 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:22.054636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:22.054667 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:19:22.054731 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:22.054738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:22.054960 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:22.055125 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:22.056593 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:22.057714 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:22.059049 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:22.059662 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:22.059791 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:22.064644 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:22.064666 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:22.064688 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:22.064734 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:22.068180 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:22.069116 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:22.069125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:22.069127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:22.069299 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:22.069703 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:22.069707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:22.069710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:22.070130 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:22.070462 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:22.070790 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 00:19:22.071101 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":345,"threads_started":1}
W20260811 00:19:22.072963 22914 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33510
W20260811 00:19:22.074231 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22679: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:19:22.074393 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22679
I20260811 00:19:22.104667 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22047
I20260811 00:19:22.105940 22151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:22.211622 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22047
2026-08-11T00:19:22Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4125 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:22 dist-test-slave-f7th krb5kdc[22951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:22 dist-test-slave-f7th krb5kdc[22951](info): set up 2 sockets
Aug 11 00:19:22 dist-test-slave-f7th krb5kdc[22951](info): commencing operation
krb5kdc: starting...
W20260811 00:19:23.494915 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:23 dist-test-slave-f7th krb5kdc[22951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:19:23.736307 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45877
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:45877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:24.087292 22969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:24.087636 22969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:24.087731 22969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:24.095603 22969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:24.095748 22969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:24.095796 22969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:24.095918 22969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:24.095963 22969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:24.106693 22969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:45877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:24.108273 22969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:24.109808 22969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:24.122505 22984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:24.122627 22979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:24.123260 22986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:24.123586 22969 server_base.cc:1034] running on GCE node
I20260811 00:19:24.124511 22969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:24.126223 22969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:24.127481 22969 hybrid_clock.cc:648] HybridClock initialized: now 1786407564127415 us; error 57 us; skew 500 ppm
I20260811 00:19:24.127915 22969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:24 dist-test-slave-f7th krb5kdc[22951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407564, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:24.131699 22969 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:19:24.133554 22969 webserver.cc:460] Webserver started at http://127.16.71.190:44693/ using document root <none> and password file <none>
I20260811 00:19:24.134325 22969 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:24.134454 22969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:24.134766 22969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:24.138115 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "795e48792e414b63b9fcc39e28df4670"
format_stamp: "Formatted at 2026-08-11 00:19:24 on dist-test-slave-f7th"
server_key: "8867b087b6391944adaa2e2ff9a088eb"
server_key_iv: "29aafb95529052f703e5bcc7f5e43c63"
server_key_version: "encryptionkey@0"
I20260811 00:19:24.138969 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "795e48792e414b63b9fcc39e28df4670"
format_stamp: "Formatted at 2026-08-11 00:19:24 on dist-test-slave-f7th"
server_key: "8867b087b6391944adaa2e2ff9a088eb"
server_key_iv: "29aafb95529052f703e5bcc7f5e43c63"
server_key_version: "encryptionkey@0"
I20260811 00:19:24.144825 22969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:19:24.148948 22994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:24.150528 22969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:19:24.150740 22969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "795e48792e414b63b9fcc39e28df4670"
format_stamp: "Formatted at 2026-08-11 00:19:24 on dist-test-slave-f7th"
server_key: "8867b087b6391944adaa2e2ff9a088eb"
server_key_iv: "29aafb95529052f703e5bcc7f5e43c63"
server_key_version: "encryptionkey@0"
I20260811 00:19:24.150959 22969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:24.163501 22969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:24.168112 22969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:24.168407 22969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:24.188056 22969 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:45877
I20260811 00:19:24.188094 23083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:45877 every 8 connection(s)
I20260811 00:19:24.189625 22969 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:24.190778 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22969
I20260811 00:19:24.191032 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:24.191838 16670 external_mini_cluster.cc:1428] Setting key a24d9aad9c13336e87800405d38aa2c1
I20260811 00:19:24.194909 23084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:19:24 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:24.209380 23084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670: Bootstrap starting.
I20260811 00:19:24.212924 23084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:24.214295 23084 log.cc:824] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:24.217614 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:24.194137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44290 (local address 127.16.71.190:45877)
0811 00:19:24.194982 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:24.194998 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:24.195054 (+ 56us) server_negotiation.cc:407] Connection header received
0811 00:19:24.196829 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:24.196855 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:24.197282 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:24.197674 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:24.199777 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:24.201369 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:24.203236 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:24.204019 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:24.204168 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:24.209356 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:24.209370 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:24.209388 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:24.209437 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:24.212725 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:24.213727 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:24.213734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:24.213741 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:24.213973 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:24.214469 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:24.214474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:24.214478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:24.215206 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:24.215548 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:24.215933 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 00:19:24.216406 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":387,"threads_started":1}
I20260811 00:19:24.217763 23084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670: No bootstrap required, opened a new log
I20260811 00:19:24.225553 23084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795e48792e414b63b9fcc39e28df4670" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45877 } }
I20260811 00:19:24.225986 23084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:24.226109 23084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795e48792e414b63b9fcc39e28df4670, State: Initialized, Role: FOLLOWER
I20260811 00:19:24.226779 23084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795e48792e414b63b9fcc39e28df4670" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45877 } }
I20260811 00:19:24.227051 23084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:24.227202 23084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:24.227389 23084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:24.229737 23084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795e48792e414b63b9fcc39e28df4670" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45877 } }
I20260811 00:19:24.230311 23084 leader_election.cc:302] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 795e48792e414b63b9fcc39e28df4670; no voters:
I20260811 00:19:24.230984 23084 leader_election.cc:288] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:24.231145 23100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:24.231813 23100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [term 1 LEADER]: Becoming Leader. State: Replica: 795e48792e414b63b9fcc39e28df4670, State: Running, Role: LEADER
I20260811 00:19:24.232376 23100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795e48792e414b63b9fcc39e28df4670" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45877 } }
I20260811 00:19:24.233242 23084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:24.235442 23103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 795e48792e414b63b9fcc39e28df4670. Latest consensus state: current_term: 1 leader_uuid: "795e48792e414b63b9fcc39e28df4670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795e48792e414b63b9fcc39e28df4670" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45877 } } }
I20260811 00:19:24.235922 23103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:24.236574 23101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "795e48792e414b63b9fcc39e28df4670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795e48792e414b63b9fcc39e28df4670" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45877 } } }
I20260811 00:19:24.237001 23101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:24.237125 23110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:24.243405 23110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:24.254263 23110 catalog_manager.cc:1347] Generated new cluster ID: b86f82d549104d5c91b00f573fe877e7
I20260811 00:19:24.254419 23110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:24.269093 23110 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:24.270500 23110 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:24.279994 23110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 795e48792e414b63b9fcc39e28df4670: Generated new TSK 0
I20260811 00:19:24.281023 23110 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:19:24.344457 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45877
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:24.701857 23125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:24.702209 23125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:24.702333 23125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:24.709867 23125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:24.710031 23125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:24.710188 23125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:24.720577 23125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:24.722018 23125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:24.723399 23125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:24.736825 23136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:24.736860 23132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:24.736894 23133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:24.738168 23125 server_base.cc:1034] running on GCE node
I20260811 00:19:24.738874 23125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:24.739874 23125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:24.741101 23125 hybrid_clock.cc:648] HybridClock initialized: now 1786407564741026 us; error 63 us; skew 500 ppm
I20260811 00:19:24.741423 23125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:24 dist-test-slave-f7th krb5kdc[22951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407564, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:24.744539 23125 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:19:24.746320 23125 webserver.cc:460] Webserver started at http://127.16.71.129:39601/ using document root <none> and password file <none>
I20260811 00:19:24.747103 23125 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:24.747237 23125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:24.747568 23125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:24.750751 23125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "cf141a96bd424c12a1e013dd85b14960"
format_stamp: "Formatted at 2026-08-11 00:19:24 on dist-test-slave-f7th"
server_key: "b36801113f3c826dcc5df74a1c3c2c53"
server_key_iv: "5867c2d78fe21f9ecb2e598aae69300c"
server_key_version: "encryptionkey@0"
I20260811 00:19:24.751608 23125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "cf141a96bd424c12a1e013dd85b14960"
format_stamp: "Formatted at 2026-08-11 00:19:24 on dist-test-slave-f7th"
server_key: "b36801113f3c826dcc5df74a1c3c2c53"
server_key_iv: "5867c2d78fe21f9ecb2e598aae69300c"
server_key_version: "encryptionkey@0"
I20260811 00:19:24.757332 23125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:19:24.761412 23146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:24.763046 23125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 00:19:24.763263 23125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "cf141a96bd424c12a1e013dd85b14960"
format_stamp: "Formatted at 2026-08-11 00:19:24 on dist-test-slave-f7th"
server_key: "b36801113f3c826dcc5df74a1c3c2c53"
server_key_iv: "5867c2d78fe21f9ecb2e598aae69300c"
server_key_version: "encryptionkey@0"
I20260811 00:19:24.763535 23125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:24.784550 23125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:24.789177 23125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:24.789490 23125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:24.790578 23125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:24.792196 23125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:24.792284 23125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:24.792414 23125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:24.792479 23125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:24.825161 23125 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:44927
I20260811 00:19:24.825222 23329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:44927 every 8 connection(s)
I20260811 00:19:24.826574 23125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:24.829248 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23125
I20260811 00:19:24.829463 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:24.830230 16670 external_mini_cluster.cc:1428] Setting key 99422b3b1516a847e677dd6036160679
Aug 11 00:19:24 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407564, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:24.850039 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:24.829414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:46135 (local address 127.16.71.190:45877)
0811 00:19:24.829921 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:24.829928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:24.831107 (+ 1179us) server_negotiation.cc:407] Connection header received
0811 00:19:24.833186 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:24.833192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:24.833284 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:24.833394 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:24.835184 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:24.836162 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:24.838124 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:24.838686 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:24.838819 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:24.842989 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:24.842995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:24.842998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:24.843041 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:24.845932 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:24.847192 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:24.847197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:24.847200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:24.847363 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:24.847983 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:24.847988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:24.847990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:24.848316 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:24.848472 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:24.849549 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0811 00:19:24.849768 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":239,"threads_started":1}
I20260811 00:19:24.851711 23333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:24.830056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45877 (local address 127.16.71.129:46135)
0811 00:19:24.830928 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:24.830988 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:24.832864 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:24.833669 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:24.833679 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:24.834272 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:24.834981 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:24.835003 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:24.836358 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:24.836362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:24.837949 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:24.837967 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:24.838962 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:24.838970 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:24.839226 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:24.839958 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:24.839988 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:24.842654 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:24.846190 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:24.846205 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:24.846230 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:24.846982 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:24.847504 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:24.847509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:24.847512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:24.847816 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:24.848466 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:24.848475 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:24.849032 (+ 557us) client_negotiation.cc:769] Sending connection context
0811 00:19:24.849395 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 00:19:24.850045 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":348,"threads_started":1}
I20260811 00:19:24.853837 23330 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45877
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
I20260811 00:19:24.854200 23330 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:24.855156 23330 heartbeater.cc:511] Master 127.16.71.190:45877 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
I20260811 00:19:24.857934 23026 ts_manager.cc:194] Registered new tserver with Master: cf141a96bd424c12a1e013dd85b14960 (127.16.71.129:44927)
I20260811 00:19:24.859526 23026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:46135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:19:24.888334 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45877
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:25.246237 23337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:25.246778 23337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:25.246968 23337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:25.256843 23337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:25.256995 23337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:25.257160 23337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:25.267889 23337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:25.269385 23337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:25.270799 23337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:25.285202 23346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:25.285406 23348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:25.285674 23342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:25.286279 23337 server_base.cc:1034] running on GCE node
I20260811 00:19:25.287026 23337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:25.288105 23337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:25.289340 23337 hybrid_clock.cc:648] HybridClock initialized: now 1786407565289275 us; error 46 us; skew 500 ppm
I20260811 00:19:25.289801 23337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:25 dist-test-slave-f7th krb5kdc[22951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407565, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:25.293450 23337 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:19:25.295142 23337 webserver.cc:460] Webserver started at http://127.16.71.130:34365/ using document root <none> and password file <none>
I20260811 00:19:25.295907 23337 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:25.296023 23337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:25.296334 23337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:25.299608 23337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "9dcc95cc996c4778a07c669bc259b4c0"
format_stamp: "Formatted at 2026-08-11 00:19:25 on dist-test-slave-f7th"
server_key: "1143f53234ba5f61cf81f5c97915e48b"
server_key_iv: "086e7a4afd7dce9edc736d1ba50b7edb"
server_key_version: "encryptionkey@0"
I20260811 00:19:25.300518 23337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "9dcc95cc996c4778a07c669bc259b4c0"
format_stamp: "Formatted at 2026-08-11 00:19:25 on dist-test-slave-f7th"
server_key: "1143f53234ba5f61cf81f5c97915e48b"
server_key_iv: "086e7a4afd7dce9edc736d1ba50b7edb"
server_key_version: "encryptionkey@0"
I20260811 00:19:25.307307 23337 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:19:25.311770 23359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:25.313752 23337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 00:19:25.313977 23337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "9dcc95cc996c4778a07c669bc259b4c0"
format_stamp: "Formatted at 2026-08-11 00:19:25 on dist-test-slave-f7th"
server_key: "1143f53234ba5f61cf81f5c97915e48b"
server_key_iv: "086e7a4afd7dce9edc736d1ba50b7edb"
server_key_version: "encryptionkey@0"
I20260811 00:19:25.314216 23337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:25.356822 23337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:25.361982 23337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:25.362304 23337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:25.363415 23337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:25.364977 23337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:25.365059 23337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:25.365164 23337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:25.365222 23337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:25.399765 23337 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:43061
I20260811 00:19:25.399849 23550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:43061 every 8 connection(s)
I20260811 00:19:25.401190 23337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:25.404528 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23337
I20260811 00:19:25.404736 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:25.405299 16670 external_mini_cluster.cc:1428] Setting key 3b69df181e90754be5abdfe3533fcea1
Aug 11 00:19:25 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407565, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:25.426419 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:25.403911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:33165 (local address 127.16.71.190:45877)
0811 00:19:25.404485 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:25.404493 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:25.405605 (+ 1112us) server_negotiation.cc:407] Connection header received
0811 00:19:25.408088 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:25.408092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:25.408164 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:25.408256 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:25.410618 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:25.411450 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:25.413891 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:25.414403 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:25.414517 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:25.419372 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:25.419378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:25.419382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:25.419426 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:25.422507 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:25.423506 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:25.423511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:25.423513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:25.423670 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:25.424215 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:25.424219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:25.424222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:25.424539 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:25.424724 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:25.425745 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0811 00:19:25.426024 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":280,"threads_started":1}
I20260811 00:19:25.427618 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:25.404461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45877 (local address 127.16.71.130:33165)
0811 00:19:25.405389 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:25.405462 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:25.407692 (+ 2230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:25.408582 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:25.408596 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:25.409425 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:25.410360 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:25.410392 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:25.411658 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:25.411664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:25.413681 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:25.413705 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:25.414677 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:25.414684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:25.415021 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:25.415885 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:25.415920 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:25.419015 (+ 3095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:25.422753 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:25.422764 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:25.422782 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:25.423335 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:25.423803 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:25.423807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:25.423810 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:25.424064 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:25.424735 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:25.424745 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:25.425263 (+ 518us) client_negotiation.cc:769] Sending connection context
0811 00:19:25.425583 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 00:19:25.426283 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":379,"threads_started":1}
I20260811 00:19:25.429286 23551 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45877
I20260811 00:19:25.429648 23551 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:19:25.430467 23551 heartbeater.cc:511] Master 127.16.71.190:45877 requested a full tablet report, sending...
I20260811 00:19:25.432168 23026 ts_manager.cc:194] Registered new tserver with Master: 9dcc95cc996c4778a07c669bc259b4c0 (127.16.71.130:43061)
I20260811 00:19:25.433216 23026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:33165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:19:25.464346 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45877
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:25.827693 23560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:25.828011 23560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:25.828142 23560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:25.836274 23560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:25.836426 23560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:25.836584 23560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:25.850553 23560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:25.854259 23560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:25.858654 23560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 00:19:25.863054 23330 heartbeater.cc:503] Master 127.16.71.190:45877 was elected leader, sending a full tablet report...
W20260811 00:19:25.873643 23567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:25.873570 23570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:25.873617 23568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:25.875070 23560 server_base.cc:1034] running on GCE node
I20260811 00:19:25.875823 23560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:25.876961 23560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:25.878225 23560 hybrid_clock.cc:648] HybridClock initialized: now 1786407565878162 us; error 48 us; skew 500 ppm
I20260811 00:19:25.878608 23560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:25 dist-test-slave-f7th krb5kdc[22951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407565, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:25.882123 23560 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:19:25.883900 23560 webserver.cc:460] Webserver started at http://127.16.71.131:45991/ using document root <none> and password file <none>
I20260811 00:19:25.884728 23560 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:25.884865 23560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:25.885196 23560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:25.888451 23560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "63ae0fa30bdf4bf298adef1482b188ca"
format_stamp: "Formatted at 2026-08-11 00:19:25 on dist-test-slave-f7th"
server_key: "28dae37b0fa7e371b816ca66eef0835b"
server_key_iv: "e851780547ae16721bf66df9a326a0c4"
server_key_version: "encryptionkey@0"
I20260811 00:19:25.889364 23560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "63ae0fa30bdf4bf298adef1482b188ca"
format_stamp: "Formatted at 2026-08-11 00:19:25 on dist-test-slave-f7th"
server_key: "28dae37b0fa7e371b816ca66eef0835b"
server_key_iv: "e851780547ae16721bf66df9a326a0c4"
server_key_version: "encryptionkey@0"
I20260811 00:19:25.895102 23560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:19:25.899001 23580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:25.900465 23560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:19:25.900689 23560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "63ae0fa30bdf4bf298adef1482b188ca"
format_stamp: "Formatted at 2026-08-11 00:19:25 on dist-test-slave-f7th"
server_key: "28dae37b0fa7e371b816ca66eef0835b"
server_key_iv: "e851780547ae16721bf66df9a326a0c4"
server_key_version: "encryptionkey@0"
I20260811 00:19:25.900911 23560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:25.947795 23560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:25.953089 23560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:25.953411 23560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:25.954483 23560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:25.956110 23560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:25.956214 23560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:25.956337 23560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:25.956395 23560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:25.992067 23560 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:46557
I20260811 00:19:25.992152 23766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:46557 every 8 connection(s)
I20260811 00:19:25.993542 23560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:26.001298 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23560
I20260811 00:19:26.001473 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:26.001971 16670 external_mini_cluster.cc:1428] Setting key 02f0c951258dc95b923ce04cc4daa971
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407565, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:26.017969 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:25.996704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:53615 (local address 127.16.71.190:45877)
0811 00:19:25.997165 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:25.997170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:25.998262 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 00:19:26.000286 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:26.000292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:26.000398 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:26.000509 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:26.002607 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.003549 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.006092 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.006574 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.006707 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:26.010842 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:26.010848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:26.010852 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:26.010891 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:26.013942 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.014953 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.014958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.014961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.015119 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.015647 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.015652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.015654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.016055 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:26.016266 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:26.017336 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0811 00:19:26.017560 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
I20260811 00:19:26.019266 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:25.997189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45877 (local address 127.16.71.131:53615)
0811 00:19:25.998065 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:25.998109 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:25.999941 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:26.000861 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:26.000872 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:26.001450 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:26.002370 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:26.002400 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.003756 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.003762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:26.005881 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:26.005901 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.006885 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.006892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:26.007150 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:26.007958 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:26.007984 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:26.010554 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:26.014178 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:26.014188 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:26.014204 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:26.014717 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:26.015239 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:26.015243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:26.015246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:26.015484 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:26.016262 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:26.016271 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:26.016860 (+ 589us) client_negotiation.cc:769] Sending connection context
0811 00:19:26.017213 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 00:19:26.017806 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":343,"threads_started":1}
I20260811 00:19:26.020754 23767 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45877
I20260811 00:19:26.021070 23767 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:26.021849 23767 heartbeater.cc:511] Master 127.16.71.190:45877 requested a full tablet report, sending...
I20260811 00:19:26.023432 23026 ts_manager.cc:194] Registered new tserver with Master: 63ae0fa30bdf4bf298adef1482b188ca (127.16.71.131:46557)
I20260811 00:19:26.024312 23026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:53615
I20260811 00:19:26.031669 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[22951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:26.092298 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:26.074772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.16.71.129:44927)
0811 00:19:26.075575 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:26.075582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:26.075611 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:19:26.075675 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:26.075681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:26.075899 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:26.076044 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:26.077763 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.078855 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.080255 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.081092 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.081209 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:26.085695 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:26.085711 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:26.085729 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:26.085773 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:26.089052 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.089957 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.089965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.089968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.090133 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.090563 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.090567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.090569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.090974 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:26.091301 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:26.091625 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 00:19:26.091914 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":448,"threads_started":1}
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:26.113247 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:26.097535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.16.71.190:45877)
0811 00:19:26.097795 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:26.097802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:26.098113 (+ 311us) server_negotiation.cc:407] Connection header received
0811 00:19:26.098299 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:26.098303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:26.098394 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:26.098486 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:26.099909 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.100721 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.102088 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.102539 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.102668 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:26.107172 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:26.107179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:26.107182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:26.107229 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:26.110167 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.111032 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.111037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.111039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.111198 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.111604 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.111608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.111610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.111985 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:26.112246 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:26.112656 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 00:19:26.112922 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 00:19:26.113895 23026 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44304
I20260811 00:19:26.115037 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23125
I20260811 00:19:26.129557 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:26.116035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.16.71.129:44927)
0811 00:19:26.116213 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:26.116218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:26.116566 (+ 348us) server_negotiation.cc:407] Connection header received
0811 00:19:26.116734 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:26.116738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:26.116864 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:26.116961 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:26.118357 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.119196 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.120740 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.121241 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.121383 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:26.123994 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:26.123999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:26.124002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:26.124045 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:26.126769 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.127530 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.127534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.127537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.127674 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.128068 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.128072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.128074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.128351 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:26.128624 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:26.128982 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 00:19:26.129228 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:19:26.130340 23320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:26.214720 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23125
I20260811 00:19:26.245172 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23337
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:19:26.268487 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:26.246320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35876 (local address 127.16.71.130:43061)
0811 00:19:26.246938 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:26.246947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:26.246974 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:19:26.247035 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:26.247041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:26.247255 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:26.247408 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:26.248738 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.249812 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.251170 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.251786 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.251916 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:26.261078 (+ 9162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:26.261097 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:26.261115 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:26.261163 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:26.264561 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.265763 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.265773 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.265776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.266009 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.266518 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.266523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.266525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.266970 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:26.267299 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:26.267732 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 00:19:26.268076 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":316,"threads_started":1}
I20260811 00:19:26.269961 23537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:26.356142 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23337
I20260811 00:19:26.378512 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23560
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[22951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:19:26.397372 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:26.379571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55440 (local address 127.16.71.131:46557)
0811 00:19:26.380184 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:26.380192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:26.380225 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:19:26.380287 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:26.380293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:26.380515 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:26.380707 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:26.382081 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.383151 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.384548 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:26.385637 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:26.385779 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:26.390847 (+ 5068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:26.390864 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:26.390881 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:26.390932 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:26.394108 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.394953 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.394961 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.394963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.395129 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:26.395554 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:26.395558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:26.395561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:26.396004 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:26.396354 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:26.396695 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 00:19:26.397001 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":271,"threads_started":1}
I20260811 00:19:26.398787 23758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:26.484771 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23560
I20260811 00:19:26.507622 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22969
I20260811 00:19:26.509104 23078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:26.618444 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22969
2026-08-11T00:19:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4417 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[23797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[23797](info): set up 2 sockets
Aug 11 00:19:26 dist-test-slave-f7th krb5kdc[23797](info): commencing operation
krb5kdc: starting...
W20260811 00:19:27.909009 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.010s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:28 dist-test-slave-f7th krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:19:28.164095 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:37135
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:37135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:28.517768 23820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:28.518103 23820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:28.518249 23820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:28.526053 23820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:28.526186 23820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:28.526245 23820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:28.526328 23820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:28.526396 23820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:28.536916 23820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:37135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:28.538295 23820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:28.539697 23820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:28.551991 23827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:28.552021 23831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:28.551994 23828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:28.553337 23820 server_base.cc:1034] running on GCE node
I20260811 00:19:28.554288 23820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:28.555871 23820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:28.557106 23820 hybrid_clock.cc:648] HybridClock initialized: now 1786407568557057 us; error 47 us; skew 500 ppm
I20260811 00:19:28.557488 23820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:28 dist-test-slave-f7th krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407568, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:28.560860 23820 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:19:28.562186 23820 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:28.562294 23820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:28.562639 23820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:28.565902 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "770a0153192646b6a1a1f2eb11514641"
format_stamp: "Formatted at 2026-08-11 00:19:28 on dist-test-slave-f7th"
server_key: "e8b0863c71b707d527773dde1f6a6d54"
server_key_iv: "d0a4a2c0dd9bec1e5865fa34a47d2a19"
server_key_version: "encryptionkey@0"
I20260811 00:19:28.566804 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "770a0153192646b6a1a1f2eb11514641"
format_stamp: "Formatted at 2026-08-11 00:19:28 on dist-test-slave-f7th"
server_key: "e8b0863c71b707d527773dde1f6a6d54"
server_key_iv: "d0a4a2c0dd9bec1e5865fa34a47d2a19"
server_key_version: "encryptionkey@0"
I20260811 00:19:28.572441 23820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:19:28.576339 23841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:28.577982 23820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:28.578194 23820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "770a0153192646b6a1a1f2eb11514641"
format_stamp: "Formatted at 2026-08-11 00:19:28 on dist-test-slave-f7th"
server_key: "e8b0863c71b707d527773dde1f6a6d54"
server_key_iv: "d0a4a2c0dd9bec1e5865fa34a47d2a19"
server_key_version: "encryptionkey@0"
I20260811 00:19:28.578444 23820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:28.603281 23820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:28.608552 23820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:28.608904 23820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:28.628943 23820 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:37135
I20260811 00:19:28.628968 23924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:37135 every 8 connection(s)
I20260811 00:19:28.629979 23820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:28.634232 23926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:28.639760 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23820
I20260811 00:19:28.639982 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:28.640702 16670 external_mini_cluster.cc:1428] Setting key c29aac165b9d2dff0d5d17f43540477e
I20260811 00:19:28.646481 23926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641: Bootstrap starting.
I20260811 00:19:28.650686 23926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:28.651860 23926 log.cc:824] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:28.655445 23926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641: No bootstrap required, opened a new log
Aug 11 00:19:28 dist-test-slave-f7th krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:28.663730 23926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770a0153192646b6a1a1f2eb11514641" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37135 } }
I20260811 00:19:28.664209 23926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:28.664309 23926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770a0153192646b6a1a1f2eb11514641, State: Initialized, Role: FOLLOWER
I20260811 00:19:28.665300 23926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770a0153192646b6a1a1f2eb11514641" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37135 } }
I20260811 00:19:28.665634 23926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:28.665800 23926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:28.666018 23926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:28.667021 23932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:28.643038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35794 (local address 127.16.71.190:37135)
0811 00:19:28.644092 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:28.644107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:28.644159 (+ 52us) server_negotiation.cc:407] Connection header received
0811 00:19:28.646111 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:28.646141 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:28.646644 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:28.647063 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:28.649268 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:28.650741 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:28.652228 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:28.653088 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:28.653282 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:28.658060 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:28.658069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:28.658091 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:28.658143 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:28.661583 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:28.662485 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:28.662493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:28.662502 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:28.662727 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:28.663146 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:28.663150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:28.663153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:28.663810 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:28.664150 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:28.664860 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 00:19:28.665421 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":520,"threads_started":1}
I20260811 00:19:28.668978 23926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770a0153192646b6a1a1f2eb11514641" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37135 } }
I20260811 00:19:28.669535 23926 leader_election.cc:302] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 770a0153192646b6a1a1f2eb11514641; no voters:
I20260811 00:19:28.670190 23926 leader_election.cc:288] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:28.670312 23935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:28.672555 23935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [term 1 LEADER]: Becoming Leader. State: Replica: 770a0153192646b6a1a1f2eb11514641, State: Running, Role: LEADER
I20260811 00:19:28.673236 23935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770a0153192646b6a1a1f2eb11514641" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37135 } }
I20260811 00:19:28.674115 23926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:28.676677 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "770a0153192646b6a1a1f2eb11514641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770a0153192646b6a1a1f2eb11514641" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37135 } } }
I20260811 00:19:28.677043 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:28.676225 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 770a0153192646b6a1a1f2eb11514641. Latest consensus state: current_term: 1 leader_uuid: "770a0153192646b6a1a1f2eb11514641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770a0153192646b6a1a1f2eb11514641" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37135 } } }
I20260811 00:19:28.677532 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:28.678423 23947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:28.683671 23947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:28.693269 23947 catalog_manager.cc:1347] Generated new cluster ID: eec1dbbda65644b388f66ba47914d8eb
I20260811 00:19:28.693403 23947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:28.705595 23947 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:28.707531 23947 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:28.715037 23947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641: Generated new TSK 0
I20260811 00:19:28.716073 23947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:19:28.785514 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37135
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:29.151021 23968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:29.151371 23968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:29.151508 23968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:29.159211 23968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:29.159373 23968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:29.159533 23968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:29.169989 23968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:29.171444 23968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:29.172900 23968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:29.186486 23976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:29.186425 23980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:29.186461 23977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:29.192085 23968 server_base.cc:1034] running on GCE node
I20260811 00:19:29.192862 23968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:29.193928 23968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:29.195156 23968 hybrid_clock.cc:648] HybridClock initialized: now 1786407569195084 us; error 64 us; skew 500 ppm
I20260811 00:19:29.195477 23968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:29 dist-test-slave-f7th krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:29.198953 23968 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:19:29.200315 23968 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:29.200419 23968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:29.200802 23968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:29.203956 23968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "08a8dd2e01cb45378b78796676567ee2"
format_stamp: "Formatted at 2026-08-11 00:19:29 on dist-test-slave-f7th"
server_key: "69ffe301590ef6c2a2748129ecfc7067"
server_key_iv: "b92866c003e8ab7ea76272fabaed19b7"
server_key_version: "encryptionkey@0"
I20260811 00:19:29.204912 23968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "08a8dd2e01cb45378b78796676567ee2"
format_stamp: "Formatted at 2026-08-11 00:19:29 on dist-test-slave-f7th"
server_key: "69ffe301590ef6c2a2748129ecfc7067"
server_key_iv: "b92866c003e8ab7ea76272fabaed19b7"
server_key_version: "encryptionkey@0"
I20260811 00:19:29.210716 23968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:19:29.214926 23991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:29.216756 23968 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:29.216964 23968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "08a8dd2e01cb45378b78796676567ee2"
format_stamp: "Formatted at 2026-08-11 00:19:29 on dist-test-slave-f7th"
server_key: "69ffe301590ef6c2a2748129ecfc7067"
server_key_iv: "b92866c003e8ab7ea76272fabaed19b7"
server_key_version: "encryptionkey@0"
I20260811 00:19:29.217209 23968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:29.245898 23968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:29.251300 23968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:29.251688 23968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:29.253150 23968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:29.255293 23968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:29.255458 23968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:29.255594 23968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:29.255672 23968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:29.288885 23968 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:32907
I20260811 00:19:29.288923 24173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:32907 every 8 connection(s)
I20260811 00:19:29.289813 23968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:29.294433 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23968
I20260811 00:19:29.294675 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:29.295553 16670 external_mini_cluster.cc:1428] Setting key 43d5c92b7324dce8885eab03c6d65a4d
Aug 11 00:19:29 dist-test-slave-f7th krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:29.313898 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:29.292496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:58691 (local address 127.16.71.190:37135)
0811 00:19:29.292991 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:29.292996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:29.294369 (+ 1373us) server_negotiation.cc:407] Connection header received
0811 00:19:29.296899 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:29.296904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:29.297028 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:29.297131 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:29.299320 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.300085 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.302415 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.302833 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.302948 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:29.307429 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:29.307437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:29.307442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:29.307484 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:29.310220 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:29.311123 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:29.311128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:29.311130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:29.311264 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:29.311855 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:29.311861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:29.311864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:29.312198 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:29.312353 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:29.313234 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 00:19:29.313513 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260811 00:19:29.314961 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:29.293120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37135 (local address 127.16.71.129:58691)
0811 00:19:29.294103 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:29.294194 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:29.296545 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:29.297462 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:29.297471 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:29.298204 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:29.299101 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:29.299130 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.300270 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.300275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:29.302211 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:29.302232 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.303100 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.303107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:29.303411 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:29.304204 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:29.304236 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:29.307101 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:29.310426 (+ 3325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:29.310436 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:29.310454 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:29.310963 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:29.311390 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:29.311394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:29.311397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:29.311633 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:29.312351 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:29.312360 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:29.312814 (+ 454us) client_negotiation.cc:769] Sending connection context
0811 00:19:29.313107 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 00:19:29.313728 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":339,"threads_started":1}
I20260811 00:19:29.317142 24174 heartbeater.cc:347] Connected to a master server at 127.16.71.190:37135
I20260811 00:19:29.317485 24174 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:29.318213 24174 heartbeater.cc:511] Master 127.16.71.190:37135 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
I20260811 00:19:29.320523 23877 ts_manager.cc:194] Registered new tserver with Master: 08a8dd2e01cb45378b78796676567ee2 (127.16.71.129:32907)
I20260811 00:19:29.322211 23877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:58691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:19:29.352538 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37135
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:29.704974 24187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:29.705299 24187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:29.705431 24187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:29.713135 24187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:29.713271 24187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:29.713421 24187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:29.724104 24187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:29.725637 24187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:29.727058 24187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:29.741024 24192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:29.740912 24196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:29.740945 24194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:29.741686 24187 server_base.cc:1034] running on GCE node
I20260811 00:19:29.742349 24187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:29.743395 24187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:29.744647 24187 hybrid_clock.cc:648] HybridClock initialized: now 1786407569744560 us; error 53 us; skew 500 ppm
I20260811 00:19:29.745005 24187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:29 dist-test-slave-f7th krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:29.748354 24187 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:19:29.749775 24187 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:29.749881 24187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:29.750174 24187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:29.753566 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "f6b37b549d0042b18a1fa73475a82924"
format_stamp: "Formatted at 2026-08-11 00:19:29 on dist-test-slave-f7th"
server_key: "c3647100ea80ccf5488c1770907a373e"
server_key_iv: "f36f901e2ddf748d3339825f0c42505d"
server_key_version: "encryptionkey@0"
I20260811 00:19:29.754448 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "f6b37b549d0042b18a1fa73475a82924"
format_stamp: "Formatted at 2026-08-11 00:19:29 on dist-test-slave-f7th"
server_key: "c3647100ea80ccf5488c1770907a373e"
server_key_iv: "f36f901e2ddf748d3339825f0c42505d"
server_key_version: "encryptionkey@0"
I20260811 00:19:29.760533 24187 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:19:29.765079 24204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:29.766782 24187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:19:29.767031 24187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "f6b37b549d0042b18a1fa73475a82924"
format_stamp: "Formatted at 2026-08-11 00:19:29 on dist-test-slave-f7th"
server_key: "c3647100ea80ccf5488c1770907a373e"
server_key_iv: "f36f901e2ddf748d3339825f0c42505d"
server_key_version: "encryptionkey@0"
I20260811 00:19:29.767277 24187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:29.790110 24187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:29.795214 24187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:29.795516 24187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:29.796646 24187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:29.798178 24187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:29.798271 24187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:29.798378 24187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:29.798444 24187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:29.834450 24187 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:39613
I20260811 00:19:29.834523 24392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:39613 every 8 connection(s)
I20260811 00:19:29.835556 24187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:29.838207 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24187
I20260811 00:19:29.838419 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:29.839071 16670 external_mini_cluster.cc:1428] Setting key e94e5b2ac0aae6df62a63d5aba501d14
Aug 11 00:19:29 dist-test-slave-f7th krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:29.861038 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:29.838933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:48699 (local address 127.16.71.190:37135)
0811 00:19:29.839463 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:29.839468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:29.841307 (+ 1839us) server_negotiation.cc:407] Connection header received
0811 00:19:29.843699 (+ 2392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:29.843706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:29.843803 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:29.843923 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:29.845930 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.846692 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.848494 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.849022 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.849154 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:29.853987 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:29.853994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:29.853998 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:29.854043 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:29.857116 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:29.858076 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:29.858081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:29.858084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:29.858239 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:29.858884 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:29.858889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:29.858891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:29.859230 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:29.859422 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:29.860299 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 00:19:29.860644 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":268,"threads_started":1}
I20260811 00:19:29.862186 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:29.839692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37135 (local address 127.16.71.130:48699)
0811 00:19:29.841067 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:29.841130 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:29.843304 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:29.844230 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:29.844241 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:29.844995 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:29.845683 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:29.845704 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.846889 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.846894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:29.848291 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:29.848309 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:29.849292 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:29.849301 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:29.849558 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:29.850415 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:29.850450 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:29.853647 (+ 3197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:29.857329 (+ 3682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:29.857339 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:29.857357 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:29.857900 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:29.858368 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:29.858371 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:29.858374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:29.858709 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:29.859427 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:29.859438 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:29.859845 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 00:19:29.860149 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 00:19:29.860839 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":507,"threads_started":1}
I20260811 00:19:29.863929 24393 heartbeater.cc:347] Connected to a master server at 127.16.71.190:37135
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:19:29.864342 24393 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:29.865211 24393 heartbeater.cc:511] Master 127.16.71.190:37135 requested a full tablet report, sending...
I20260811 00:19:29.866971 23877 ts_manager.cc:194] Registered new tserver with Master: f6b37b549d0042b18a1fa73475a82924 (127.16.71.130:39613)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
I20260811 00:19:29.867872 23877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:48699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:19:29.898504 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37135
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:30.251389 24403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:30.251775 24403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:30.251890 24403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:30.259371 24403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:30.259521 24403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:30.259642 24403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:30.270037 24403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:30.271489 24403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:30.272976 24403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:30.286948 24413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:30.286983 24412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:30.286983 24416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:30.288304 24403 server_base.cc:1034] running on GCE node
I20260811 00:19:30.289136 24403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:30.290238 24403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:30.291491 24403 hybrid_clock.cc:648] HybridClock initialized: now 1786407570291442 us; error 55 us; skew 500 ppm
I20260811 00:19:30.291864 24403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:30 dist-test-slave-f7th krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:30.295445 24403 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:19:30.296836 24403 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:30.296959 24403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:30.297282 24403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:30.300457 24403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "92efbef26a59464cba8ceca7d1095fed"
format_stamp: "Formatted at 2026-08-11 00:19:30 on dist-test-slave-f7th"
server_key: "13cf996371105b5b18f0d114b5881987"
server_key_iv: "2c43101f32182f8c0da56cb5dbf34ab7"
server_key_version: "encryptionkey@0"
I20260811 00:19:30.301494 24403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "92efbef26a59464cba8ceca7d1095fed"
format_stamp: "Formatted at 2026-08-11 00:19:30 on dist-test-slave-f7th"
server_key: "13cf996371105b5b18f0d114b5881987"
server_key_iv: "2c43101f32182f8c0da56cb5dbf34ab7"
server_key_version: "encryptionkey@0"
I20260811 00:19:30.307518 24403 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:19:30.311811 24428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:30.313469 24403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:30.313689 24403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "92efbef26a59464cba8ceca7d1095fed"
format_stamp: "Formatted at 2026-08-11 00:19:30 on dist-test-slave-f7th"
server_key: "13cf996371105b5b18f0d114b5881987"
server_key_iv: "2c43101f32182f8c0da56cb5dbf34ab7"
server_key_version: "encryptionkey@0"
I20260811 00:19:30.313935 24403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:30.325806 24174 heartbeater.cc:503] Master 127.16.71.190:37135 was elected leader, sending a full tablet report...
I20260811 00:19:30.329437 24403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:30.334015 24403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:30.334313 24403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:30.335407 24403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:30.337100 24403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:30.337195 24403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:30.337311 24403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:30.337378 24403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:30.369772 24403 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:33681
I20260811 00:19:30.369837 24618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:33681 every 8 connection(s)
I20260811 00:19:30.370698 24403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:30.374138 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24403
I20260811 00:19:30.374313 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:30.374771 16670 external_mini_cluster.cc:1428] Setting key 39e5b3495b3a717132dafb3e9fa233ad
Aug 11 00:19:30 dist-test-slave-f7th krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:30.396893 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.373941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37135 (local address 127.16.71.131:49553)
0811 00:19:30.374799 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:30.374846 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:30.376859 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:30.377575 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:30.377585 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:30.378364 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:30.379164 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.379198 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.380744 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.380752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.382848 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.382870 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.383946 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.383956 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.384307 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.385118 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:30.385154 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:30.388108 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:30.392027 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:30.392039 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:30.392063 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:30.392897 (+ 834us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:30.393367 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:30.393373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:30.393377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:30.393679 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:30.394352 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:30.394391 (+ 39us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:30.394809 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 00:19:30.395110 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 00:19:30.395636 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":363,"threads_started":1}
I20260811 00:19:30.397562 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.373335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:49553 (local address 127.16.71.190:37135)
0811 00:19:30.373822 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.373829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.374976 (+ 1147us) server_negotiation.cc:407] Connection header received
0811 00:19:30.377166 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.377171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.377259 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.377378 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:30.379397 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.380503 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.383087 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.383691 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.383819 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.388418 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:30.388426 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:30.388432 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:30.388484 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:30.391825 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.393063 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.393069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.393071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.393225 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.393809 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.393814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.393818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.394195 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:30.394353 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.396903 (+ 2550us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.397197 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":206,"threads_started":1}
I20260811 00:19:30.398872 24619 heartbeater.cc:347] Connected to a master server at 127.16.71.190:37135
I20260811 00:19:30.399228 24619 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:30.400146 24619 heartbeater.cc:511] Master 127.16.71.190:37135 requested a full tablet report, sending...
I20260811 00:19:30.401980 23877 ts_manager.cc:194] Registered new tserver with Master: 92efbef26a59464cba8ceca7d1095fed (127.16.71.131:33681)
I20260811 00:19:30.402925 23877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:49553
I20260811 00:19:30.405007 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:19:30.424510 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.410222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.16.71.190:37135)
0811 00:19:30.410418 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.410423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.411022 (+ 599us) server_negotiation.cc:407] Connection header received
0811 00:19:30.411145 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.411149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.411226 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.411327 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:30.413087 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.413850 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.415228 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.415767 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.415891 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.418634 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:30.418642 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:30.418646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:30.418701 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:30.421584 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.422451 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.422455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.422457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.422604 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.422997 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.423001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.423004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.423378 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:30.423610 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.424036 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.424253 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:19:30.429382 23876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:30.436570 23876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:30.474051 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.451321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.16.71.129:32907)
0811 00:19:30.464851 (+ 13530us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.464861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.464898 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:19:30.465014 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.465022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.465320 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.465582 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:30.466961 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.468753 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.471573 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.472780 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.472922 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.473192 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.473505 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.473684 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13328,"thread_start_us":13080,"threads_started":1}
I20260811 00:19:30.474699 24636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.451031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:32907 (local address 127.0.0.1:48866)
0811 00:19:30.463276 (+ 12245us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:30.463350 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:30.463639 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:30.465587 (+ 1948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:30.465595 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:30.465626 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:30.466668 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.466685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.468917 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.468926 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.471324 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.471350 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.472991 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.472997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.473229 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.473291 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 00:19:30.473427 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 00:19:30.473666 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11839,"mutex_wait_us":167,"thread_start_us":13807,"threads_started":1}
I20260811 00:19:30.475098 24635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.451287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:33681 (local address 127.0.0.1:45356)
0811 00:19:30.465154 (+ 13867us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:30.465175 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:30.465307 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:30.465993 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:30.465997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:30.466023 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:30.466672 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.466689 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.468871 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.468876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.471403 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.471420 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.473612 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.473619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.473824 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.473862 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:19:30.473929 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 00:19:30.474106 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13725,"mutex_wait_us":48,"spinlock_wait_cycles":11136,"thread_start_us":12639,"threads_started":1}
I20260811 00:19:30.475828 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.451567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.16.71.131:33681)
0811 00:19:30.463946 (+ 12379us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.463956 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.465395 (+ 1439us) server_negotiation.cc:407] Connection header received
0811 00:19:30.465499 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.465506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.465752 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.465983 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:30.467082 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.468726 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.471756 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.473364 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.474283 (+ 919us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.474540 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.474764 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.475485 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12151,"thread_start_us":12407,"threads_started":1}
I20260811 00:19:30.476564 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.465301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.16.71.130:39613)
0811 00:19:30.466161 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.466169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.467396 (+ 1227us) server_negotiation.cc:407] Connection header received
0811 00:19:30.467512 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.467520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.467778 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.467974 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:30.468727 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.470264 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.473001 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.474677 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.474814 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.475133 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.476056 (+ 923us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.476236 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":547,"threads_started":1}
I20260811 00:19:30.477128 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.465243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:39613 (local address 127.0.0.1:43968)
0811 00:19:30.467140 (+ 1897us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:30.467156 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:30.467274 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:30.468198 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:30.468202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:30.468228 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:30.468543 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.468552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.470463 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.470470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.472784 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.472802 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.475292 (+ 2490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.475299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.475478 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.475510 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:19:30.475602 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 00:19:30.475741 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1779,"mutex_wait_us":3,"thread_start_us":1628,"threads_started":1}
I20260811 00:19:30.479378 24060 tablet_service.cc:1467] Processing CreateTablet for tablet 691c7ea58ce445c3af756a966c9221be (DEFAULT_TABLE table=test-table [id=645e1fa490684554ae8c57e92ca3e1af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:30.480047 24286 tablet_service.cc:1467] Processing CreateTablet for tablet 691c7ea58ce445c3af756a966c9221be (DEFAULT_TABLE table=test-table [id=645e1fa490684554ae8c57e92ca3e1af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:30.481427 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691c7ea58ce445c3af756a966c9221be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:30.480100 24498 tablet_service.cc:1467] Processing CreateTablet for tablet 691c7ea58ce445c3af756a966c9221be (DEFAULT_TABLE table=test-table [id=645e1fa490684554ae8c57e92ca3e1af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:30.481683 24060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691c7ea58ce445c3af756a966c9221be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:30.482424 24498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691c7ea58ce445c3af756a966c9221be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:30.494352 24643 tablet_bootstrap.cc:492] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Bootstrap starting.
I20260811 00:19:30.497970 24648 tablet_bootstrap.cc:492] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Bootstrap starting.
I20260811 00:19:30.499326 24643 tablet_bootstrap.cc:654] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:30.500824 24643 log.cc:824] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:30.501519 24646 tablet_bootstrap.cc:492] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Bootstrap starting.
I20260811 00:19:30.502187 24648 tablet_bootstrap.cc:654] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:30.503564 24648 log.cc:824] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:30.505888 24643 tablet_bootstrap.cc:492] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: No bootstrap required, opened a new log
I20260811 00:19:30.505859 24646 tablet_bootstrap.cc:654] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:30.506230 24643 ts_tablet_manager.cc:1397] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260811 00:19:30.507356 24646 log.cc:824] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:30.507808 24648 tablet_bootstrap.cc:492] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: No bootstrap required, opened a new log
I20260811 00:19:30.508126 24648 ts_tablet_manager.cc:1397] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 00:19:30.511201 24646 tablet_bootstrap.cc:492] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: No bootstrap required, opened a new log
I20260811 00:19:30.511554 24646 ts_tablet_manager.cc:1397] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 00:19:30.516448 24648 raft_consensus.cc:348] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.516947 24648 raft_consensus.cc:374] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:30.516734 24643 raft_consensus.cc:348] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.517061 24648 raft_consensus.cc:729] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92efbef26a59464cba8ceca7d1095fed, State: Initialized, Role: FOLLOWER
I20260811 00:19:30.517189 24643 raft_consensus.cc:374] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:30.517295 24643 raft_consensus.cc:729] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b37b549d0042b18a1fa73475a82924, State: Initialized, Role: FOLLOWER
I20260811 00:19:30.517761 24648 consensus_queue.cc:260] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.517920 24643 consensus_queue.cc:260] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.519153 24619 heartbeater.cc:503] Master 127.16.71.190:37135 was elected leader, sending a full tablet report...
I20260811 00:19:30.519640 24648 ts_tablet_manager.cc:1428] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 00:19:30.523342 24646 raft_consensus.cc:348] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.523885 24646 raft_consensus.cc:374] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:30.523995 24646 raft_consensus.cc:729] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a8dd2e01cb45378b78796676567ee2, State: Initialized, Role: FOLLOWER
I20260811 00:19:30.524751 24646 consensus_queue.cc:260] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.526028 24393 heartbeater.cc:503] Master 127.16.71.190:37135 was elected leader, sending a full tablet report...
I20260811 00:19:30.526990 24643 ts_tablet_manager.cc:1428] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Time spent starting tablet: real 0.021s user 0.011s sys 0.010s
I20260811 00:19:30.527037 24646 ts_tablet_manager.cc:1428] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260811 00:19:30.542366 24175 tablet.cc:2369] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:30.568315 24654 raft_consensus.cc:482] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:30.568737 24654 raft_consensus.cc:504] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.570564 24654 leader_election.cc:288] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08a8dd2e01cb45378b78796676567ee2 (127.16.71.129:32907), f6b37b549d0042b18a1fa73475a82924 (127.16.71.130:39613)
I20260811 00:19:30.576979 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.571002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:60017 (local address 127.16.71.129:32907)
0811 00:19:30.571296 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.571302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.571539 (+ 237us) server_negotiation.cc:407] Connection header received
0811 00:19:30.571740 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.571745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.571828 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.571921 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:30.572328 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.573187 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.574972 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.576098 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.576257 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.576333 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.576593 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.576749 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 00:19:30.576990 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.571026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:32907 (local address 127.16.71.131:60017)
0811 00:19:30.571420 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:30.571437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:30.571596 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:30.571962 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:30.571967 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:30.571996 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:30.572189 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.572198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.573338 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.573343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.574810 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.574827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.576250 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.576255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.576396 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.576424 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:19:30.576510 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 00:19:30.576680 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":30}
I20260811 00:19:30.577878 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.571032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:33991 (local address 127.16.71.130:39613)
0811 00:19:30.571286 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.571292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.572010 (+ 718us) server_negotiation.cc:407] Connection header received
0811 00:19:30.572150 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.572154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.572231 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.572320 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:30.572918 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.573640 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.575666 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.576810 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.576954 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.577025 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.577478 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.577645 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:19:30.577920 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.571207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:39613 (local address 127.16.71.131:33991)
0811 00:19:30.571892 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:30.571908 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:30.572026 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:30.572344 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:30.572349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:30.572377 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:30.572751 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.572763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.573807 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.573812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.575480 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:30.575497 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.577027 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.577034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:30.577291 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.577328 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:19:30.577412 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 00:19:30.577546 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":3,"thread_start_us":395,"threads_started":1}
I20260811 00:19:30.578248 24094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691c7ea58ce445c3af756a966c9221be" candidate_uuid: "92efbef26a59464cba8ceca7d1095fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a8dd2e01cb45378b78796676567ee2" is_pre_election: true
I20260811 00:19:30.578766 24094 raft_consensus.cc:2454] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92efbef26a59464cba8ceca7d1095fed in term 0.
I20260811 00:19:30.579027 24313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691c7ea58ce445c3af756a966c9221be" candidate_uuid: "92efbef26a59464cba8ceca7d1095fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b37b549d0042b18a1fa73475a82924" is_pre_election: true
I20260811 00:19:30.579495 24430 leader_election.cc:302] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a8dd2e01cb45378b78796676567ee2, 92efbef26a59464cba8ceca7d1095fed; no voters:
I20260811 00:19:30.579571 24313 raft_consensus.cc:2454] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92efbef26a59464cba8ceca7d1095fed in term 0.
I20260811 00:19:30.579974 24654 raft_consensus.cc:2781] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:30.580150 24654 raft_consensus.cc:482] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:30.580264 24654 raft_consensus.cc:3037] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:30.582607 24654 raft_consensus.cc:504] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.583232 24654 leader_election.cc:288] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [CANDIDATE]: Term 1 election: Requested vote from peers 08a8dd2e01cb45378b78796676567ee2 (127.16.71.129:32907), f6b37b549d0042b18a1fa73475a82924 (127.16.71.130:39613)
I20260811 00:19:30.583753 24313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691c7ea58ce445c3af756a966c9221be" candidate_uuid: "92efbef26a59464cba8ceca7d1095fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b37b549d0042b18a1fa73475a82924"
I20260811 00:19:30.583989 24313 raft_consensus.cc:3037] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:30.584039 24094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691c7ea58ce445c3af756a966c9221be" candidate_uuid: "92efbef26a59464cba8ceca7d1095fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a8dd2e01cb45378b78796676567ee2"
I20260811 00:19:30.584259 24094 raft_consensus.cc:3037] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:30.587553 24313 raft_consensus.cc:2454] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92efbef26a59464cba8ceca7d1095fed in term 1.
I20260811 00:19:30.587592 24094 raft_consensus.cc:2454] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92efbef26a59464cba8ceca7d1095fed in term 1.
W20260811 00:19:30.587963 24394 tablet.cc:2369] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:30.588142 24430 leader_election.cc:302] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a8dd2e01cb45378b78796676567ee2, 92efbef26a59464cba8ceca7d1095fed; no voters:
I20260811 00:19:30.588483 24654 raft_consensus.cc:2781] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:30.589056 24654 raft_consensus.cc:686] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 1 LEADER]: Becoming Leader. State: Replica: 92efbef26a59464cba8ceca7d1095fed, State: Running, Role: LEADER
I20260811 00:19:30.589576 24654 consensus_queue.cc:237] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } }
I20260811 00:19:30.594441 23876 catalog_manager.cc:5495] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed reported cstate change: term changed from 0 to 1, leader changed from <none> to 92efbef26a59464cba8ceca7d1095fed (127.16.71.131). New cstate: current_term: 1 leader_uuid: "92efbef26a59464cba8ceca7d1095fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92efbef26a59464cba8ceca7d1095fed" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:30.613533 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.607710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.16.71.131:33681)
0811 00:19:30.607945 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.607951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.608072 (+ 121us) server_negotiation.cc:407] Connection header received
0811 00:19:30.608281 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.608285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.608388 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.608525 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:30.609013 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.609730 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.611216 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.611645 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.611760 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.612287 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:30.612934 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:30.613086 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.613195 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.613278 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 00:19:30.622306 24620 tablet.cc:2369] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:30.622867 24094 raft_consensus.cc:1261] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 1 FOLLOWER]: Refusing update from remote peer 92efbef26a59464cba8ceca7d1095fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:30.622897 24313 raft_consensus.cc:1261] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 1 FOLLOWER]: Refusing update from remote peer 92efbef26a59464cba8ceca7d1095fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:30.623942 24660 consensus_queue.cc:1035] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b37b549d0042b18a1fa73475a82924" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:30.624362 24654 consensus_queue.cc:1035] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "08a8dd2e01cb45378b78796676567ee2" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:30.656734 24669 mvcc.cc:205] Tried to move back new op lower bound from 7317125409259831296 to 7317125409139048448. Current Snapshot: MvccSnapshot[applied={T|T < 7317125409259831296}]
I20260811 00:19:30.659441 23876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:30.659740 23876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:30.664162 23876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 770a0153192646b6a1a1f2eb11514641: Sending DeleteTablet for 3 replicas of tablet 691c7ea58ce445c3af756a966c9221be
I20260811 00:19:30.665616 24060 tablet_service.cc:1514] Processing DeleteTablet for tablet 691c7ea58ce445c3af756a966c9221be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:30 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:48866
I20260811 00:19:30.666404 24286 tablet_service.cc:1514] Processing DeleteTablet for tablet 691c7ea58ce445c3af756a966c9221be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:30 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:43968
I20260811 00:19:30.666496 24498 tablet_service.cc:1514] Processing DeleteTablet for tablet 691c7ea58ce445c3af756a966c9221be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:30 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:45356
I20260811 00:19:30.666913 24683 tablet_replica.cc:330] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: stopping tablet replica
I20260811 00:19:30.667447 24683 raft_consensus.cc:2229] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:30.668087 24683 raft_consensus.cc:2258] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:30.668496 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23968
I20260811 00:19:30.669114 24690 tablet_replica.cc:330] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: stopping tablet replica
I20260811 00:19:30.669624 24690 raft_consensus.cc:2229] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:30.670266 24690 raft_consensus.cc:2258] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:30.672797 24690 ts_tablet_manager.cc:1905] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:30.672891 24683 ts_tablet_manager.cc:1905] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:30.680756 24690 ts_tablet_manager.cc:1918] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:30.681102 24690 log.cc:1196] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/691c7ea58ce445c3af756a966c9221be
I20260811 00:19:30.681767 24690 ts_tablet_manager.cc:1939] T 691c7ea58ce445c3af756a966c9221be P f6b37b549d0042b18a1fa73475a82924: Deleting consensus metadata
I20260811 00:19:30.684180 23851 catalog_manager.cc:4841] TS f6b37b549d0042b18a1fa73475a82924 (127.16.71.130:39613): tablet 691c7ea58ce445c3af756a966c9221be (table test-table [id=645e1fa490684554ae8c57e92ca3e1af]) successfully deleted
I20260811 00:19:30.686540 24683 ts_tablet_manager.cc:1918] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:30.686894 24683 log.cc:1196] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/691c7ea58ce445c3af756a966c9221be
I20260811 00:19:30.687553 24683 ts_tablet_manager.cc:1939] T 691c7ea58ce445c3af756a966c9221be P 08a8dd2e01cb45378b78796676567ee2: Deleting consensus metadata
I20260811 00:19:30.689841 23845 catalog_manager.cc:4841] TS 08a8dd2e01cb45378b78796676567ee2 (127.16.71.129:32907): tablet 691c7ea58ce445c3af756a966c9221be (table test-table [id=645e1fa490684554ae8c57e92ca3e1af]) successfully deleted
I20260811 00:19:30.691437 24684 tablet_replica.cc:330] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: stopping tablet replica
I20260811 00:19:30.691967 24684 raft_consensus.cc:2229] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:30.692767 24684 raft_consensus.cc:2258] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:30.695020 24684 ts_tablet_manager.cc:1905] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:19:30 dist-test-slave-f7th krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:30.703119 24684 ts_tablet_manager.cc:1918] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:30.703329 24684 log.cc:1196] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/691c7ea58ce445c3af756a966c9221be
I20260811 00:19:30.703926 24684 ts_tablet_manager.cc:1939] T 691c7ea58ce445c3af756a966c9221be P 92efbef26a59464cba8ceca7d1095fed: Deleting consensus metadata
I20260811 00:19:30.705807 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.669821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.16.71.129:32907)
0811 00:19:30.670006 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.670012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.690695 (+ 20683us) server_negotiation.cc:407] Connection header received
0811 00:19:30.690897 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.690903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.691004 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.691122 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:30.692642 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.693629 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.694907 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.695388 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.695515 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.699858 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:30.699876 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:30.699892 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:30.699939 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:30.702754 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.703558 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.703568 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.703572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.703732 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.704121 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.704126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.704130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.704553 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:30.704905 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.705197 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.705438 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:19:30.706063 23851 catalog_manager.cc:4841] TS 92efbef26a59464cba8ceca7d1095fed (127.16.71.131:33681): tablet 691c7ea58ce445c3af756a966c9221be (table test-table [id=645e1fa490684554ae8c57e92ca3e1af]) successfully deleted
I20260811 00:19:30.706508 24167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:30.818214 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23968
I20260811 00:19:30.850888 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24187
Aug 11 00:19:30 dist-test-slave-f7th krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:19:30.872402 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.851883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38092 (local address 127.16.71.130:39613)
0811 00:19:30.852065 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.852071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.852097 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:19:30.852320 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.852324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.852396 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.852654 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:30.854100 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.854954 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.856286 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.856896 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:30.857030 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:30.863114 (+ 6084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:30.863135 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:30.863158 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:30.863223 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:30.868278 (+ 5055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.869270 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.869279 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.869283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.869459 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:30.869891 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:30.869895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:30.869897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:30.870365 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:30.870653 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:30.871834 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0811 00:19:30.872076 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:19:30.874686 24384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:30.969779 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24187
I20260811 00:19:30.994233 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24403
Aug 11 00:19:31 dist-test-slave-f7th krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:19:31.015750 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:30.995079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.16.71.131:33681)
0811 00:19:30.995258 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:30.995262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:30.995316 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:19:30.995649 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:30.995654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:30.995727 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:30.995836 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:30.997885 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:30.998812 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:31.001599 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:31.002135 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:31.002259 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:31.008326 (+ 6067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:31.008343 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:31.008365 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:31.008423 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:31.011929 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:31.013130 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:31.013140 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:31.013142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:31.013364 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:31.013864 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:31.013868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:31.013871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:31.014351 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:31.014721 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:31.015067 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 00:19:31.015338 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:19:31.016708 24611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:31.140916 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24403
I20260811 00:19:31.168401 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23820
I20260811 00:19:31.169570 23920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:31.282305 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23820
2026-08-11T00:19:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4655 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T00:19:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:31Z Disabled control of system clock
I20260811 00:19:31.365737 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:35293
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:45489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:35293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 00:19:31.741565 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:31.741915 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:31.742010 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:31.749809 24706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:31.749964 24706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:31.750003 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:31.750054 24706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:31.750099 24706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:31.759939 24706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:35293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:35293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:31.761354 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:31.763275 24706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:31.775542 24716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:31.775484 24724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:31.775516 24720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:31.776351 24706 server_base.cc:1034] running on GCE node
I20260811 00:19:31.777272 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:31.778864 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:31.780134 24706 hybrid_clock.cc:648] HybridClock initialized: now 1786407571780080 us; error 44 us; skew 500 ppm
I20260811 00:19:31.780592 24706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:31.783335 24706 webserver.cc:460] Webserver started at http://127.16.71.190:46179/ using document root <none> and password file <none>
I20260811 00:19:31.784121 24706 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:31.784231 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:31.784581 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:31.787864 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "8e662ab40f214793879ed9a420189049"
format_stamp: "Formatted at 2026-08-11 00:19:31 on dist-test-slave-f7th"
server_key: "e834b4f1bd1c30501db5348dc31a2c97"
server_key_iv: "230ea5f2441029ed88e44bfe91e3f7aa"
server_key_version: "encryptionkey@0"
I20260811 00:19:31.789006 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "8e662ab40f214793879ed9a420189049"
format_stamp: "Formatted at 2026-08-11 00:19:31 on dist-test-slave-f7th"
server_key: "e834b4f1bd1c30501db5348dc31a2c97"
server_key_iv: "230ea5f2441029ed88e44bfe91e3f7aa"
server_key_version: "encryptionkey@0"
I20260811 00:19:31.795610 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:19:31.801141 24733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:31.803303 24706 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 00:19:31.803541 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "8e662ab40f214793879ed9a420189049"
format_stamp: "Formatted at 2026-08-11 00:19:31 on dist-test-slave-f7th"
server_key: "e834b4f1bd1c30501db5348dc31a2c97"
server_key_iv: "230ea5f2441029ed88e44bfe91e3f7aa"
server_key_version: "encryptionkey@0"
I20260811 00:19:31.803807 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:31.811551 24706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:31.812605 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:31.813179 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:31.835052 24706 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:35293
I20260811 00:19:31.835065 24814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:35293 every 8 connection(s)
I20260811 00:19:31.836568 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:31.841125 24820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:31.841919 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24706
I20260811 00:19:31.842180 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:31.842850 16670 external_mini_cluster.cc:1428] Setting key c21e9edb97361a7a379f1ea7e93006bd
I20260811 00:19:31.854058 24827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:31.845339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.16.71.190:35293)
0811 00:19:31.846718 (+ 1379us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:31.846737 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:31.846842 (+ 105us) server_negotiation.cc:407] Connection header received
0811 00:19:31.848662 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:31.848700 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:31.849443 (+ 743us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:31.850028 (+ 585us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:31.851631 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:31.851641 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:31.851660 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:31.851683 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:31.851776 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:31.851802 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:31.852061 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:31.852205 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 00:19:31.852526 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":400,"threads_started":1}
I20260811 00:19:31.857633 24820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049: Bootstrap starting.
I20260811 00:19:31.862017 24820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:31.863282 24820 log.cc:824] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:31.867739 24820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049: No bootstrap required, opened a new log
I20260811 00:19:31.875701 24820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e662ab40f214793879ed9a420189049" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35293 } }
I20260811 00:19:31.876186 24820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:31.876296 24820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e662ab40f214793879ed9a420189049, State: Initialized, Role: FOLLOWER
I20260811 00:19:31.877014 24820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e662ab40f214793879ed9a420189049" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35293 } }
I20260811 00:19:31.877267 24820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:31.877403 24820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:31.877570 24820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:31.879922 24820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e662ab40f214793879ed9a420189049" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35293 } }
I20260811 00:19:31.880539 24820 leader_election.cc:302] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e662ab40f214793879ed9a420189049; no voters:
I20260811 00:19:31.881421 24820 leader_election.cc:288] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:31.881574 24829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:31.882278 24829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [term 1 LEADER]: Becoming Leader. State: Replica: 8e662ab40f214793879ed9a420189049, State: Running, Role: LEADER
I20260811 00:19:31.882879 24829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e662ab40f214793879ed9a420189049" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35293 } }
I20260811 00:19:31.883694 24820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:31.885782 24832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e662ab40f214793879ed9a420189049. Latest consensus state: current_term: 1 leader_uuid: "8e662ab40f214793879ed9a420189049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e662ab40f214793879ed9a420189049" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35293 } } }
I20260811 00:19:31.885933 24830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e662ab40f214793879ed9a420189049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e662ab40f214793879ed9a420189049" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35293 } } }
I20260811 00:19:31.886225 24832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:31.886307 24830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:31.889863 24839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:31.895782 24839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:31.905563 24839 catalog_manager.cc:1347] Generated new cluster ID: a0e1436900f64ab5bfaec1b4ed243245
I20260811 00:19:31.905709 24839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:31.936581 24839 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:31.937150 24839 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:31.944940 24839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049: Generated new TSK 0
I20260811 00:19:31.946223 24839 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:19:31.959815 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:35293
--builtin_ntp_servers=127.16.71.148:45489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 00:19:32.315862 24854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:32.316185 24854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:32.316296 24854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:32.324186 24854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:32.324302 24854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:32.324460 24854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:32.334544 24854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:35293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:32.335930 24854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:32.337842 24854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:32.351295 24860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:32.351269 24861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:32.351351 24867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:32.352679 24854 server_base.cc:1034] running on GCE node
I20260811 00:19:32.353394 24854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:32.354429 24854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:32.355696 24854 hybrid_clock.cc:648] HybridClock initialized: now 1786407572355612 us; error 58 us; skew 500 ppm
I20260811 00:19:32.356092 24854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:32.358673 24854 webserver.cc:460] Webserver started at http://127.16.71.129:37307/ using document root <none> and password file <none>
I20260811 00:19:32.359441 24854 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:32.359543 24854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:32.359905 24854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:32.363171 24854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "3f5b71001ff04c9ab7abc6050942c441"
format_stamp: "Formatted at 2026-08-11 00:19:32 on dist-test-slave-f7th"
server_key: "d4a55548a9fa71601d0dacec1f37f4a6"
server_key_iv: "3098b0431c7c582d9b110baf894e316a"
server_key_version: "encryptionkey@0"
I20260811 00:19:32.364025 24854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "3f5b71001ff04c9ab7abc6050942c441"
format_stamp: "Formatted at 2026-08-11 00:19:32 on dist-test-slave-f7th"
server_key: "d4a55548a9fa71601d0dacec1f37f4a6"
server_key_iv: "3098b0431c7c582d9b110baf894e316a"
server_key_version: "encryptionkey@0"
I20260811 00:19:32.369684 24854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 00:19:32.373551 24880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:32.375085 24854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:19:32.375275 24854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "3f5b71001ff04c9ab7abc6050942c441"
format_stamp: "Formatted at 2026-08-11 00:19:32 on dist-test-slave-f7th"
server_key: "d4a55548a9fa71601d0dacec1f37f4a6"
server_key_iv: "3098b0431c7c582d9b110baf894e316a"
server_key_version: "encryptionkey@0"
I20260811 00:19:32.375519 24854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:32.381687 24854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:32.382560 24854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:32.382825 24854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:32.383826 24854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:32.385596 24854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:32.385697 24854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:32.385816 24854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:32.385886 24854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:32.424037 24854 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:37837
I20260811 00:19:32.424100 25075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:37837 every 8 connection(s)
I20260811 00:19:32.425472 24854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:32.434477 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:32.428242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:60759 (local address 127.16.71.190:35293)
0811 00:19:32.428697 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:32.428702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:32.430048 (+ 1346us) server_negotiation.cc:407] Connection header received
0811 00:19:32.431822 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:32.431829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:32.431903 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:32.432002 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:32.433591 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:32.433596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:32.433600 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:32.433619 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:32.433667 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:32.433681 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:32.433824 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:32.434030 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 00:19:32.434108 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":212,"threads_started":1}
I20260811 00:19:32.435279 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24854
I20260811 00:19:32.435516 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:32.435530 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:32.428886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:35293 (local address 127.16.71.129:60759)
0811 00:19:32.429872 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:32.429911 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:32.431516 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:32.432271 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:32.432281 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:32.432385 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:32.433205 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:32.433208 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:32.433254 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:32.433365 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:32.433787 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:32.433800 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:32.433808 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:19:32.434121 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 00:19:32.434428 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":282,"threads_started":1}
I20260811 00:19:32.436285 16670 external_mini_cluster.cc:1428] Setting key fe8f7f6283d05b4a372786c6351dde8c
I20260811 00:19:32.437507 25076 heartbeater.cc:347] Connected to a master server at 127.16.71.190:35293
I20260811 00:19:32.437880 25076 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:32.438594 25076 heartbeater.cc:511] Master 127.16.71.190:35293 requested a full tablet report, sending...
I20260811 00:19:32.439888 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:35293
--builtin_ntp_servers=127.16.71.148:45489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 00:19:32.440999 24763 ts_manager.cc:194] Registered new tserver with Master: 3f5b71001ff04c9ab7abc6050942c441 (127.16.71.129:37837)
W20260811 00:19:32.793040 25082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:32.793352 25082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:32.793474 25082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:32.801286 25082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:32.801424 25082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:32.801563 25082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:32.811364 25082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:35293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:32.812829 25082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:32.814858 25082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:32.829640 25089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:32.829682 25088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:32.829603 25091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:32.830515 25082 server_base.cc:1034] running on GCE node
I20260811 00:19:32.831257 25082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:32.832551 25082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:32.833844 25082 hybrid_clock.cc:648] HybridClock initialized: now 1786407572833783 us; error 54 us; skew 500 ppm
I20260811 00:19:32.834241 25082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:32.837000 25082 webserver.cc:460] Webserver started at http://127.16.71.130:37833/ using document root <none> and password file <none>
I20260811 00:19:32.837814 25082 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:32.837951 25082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:32.838328 25082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:32.841718 25082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "d3923f52c8594666916e5e4e304e21b5"
format_stamp: "Formatted at 2026-08-11 00:19:32 on dist-test-slave-f7th"
server_key: "bf2386262b71ea53268b56ad4841f382"
server_key_iv: "2d799cc339c62f299187b9c61add3421"
server_key_version: "encryptionkey@0"
I20260811 00:19:32.842774 25082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "d3923f52c8594666916e5e4e304e21b5"
format_stamp: "Formatted at 2026-08-11 00:19:32 on dist-test-slave-f7th"
server_key: "bf2386262b71ea53268b56ad4841f382"
server_key_iv: "2d799cc339c62f299187b9c61add3421"
server_key_version: "encryptionkey@0"
I20260811 00:19:32.849213 25082 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:19:32.853459 25111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:32.855114 25082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:19:32.855329 25082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "d3923f52c8594666916e5e4e304e21b5"
format_stamp: "Formatted at 2026-08-11 00:19:32 on dist-test-slave-f7th"
server_key: "bf2386262b71ea53268b56ad4841f382"
server_key_iv: "2d799cc339c62f299187b9c61add3421"
server_key_version: "encryptionkey@0"
I20260811 00:19:32.855590 25082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:32.862270 25082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:32.863273 25082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:32.863579 25082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:32.864722 25082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:32.866640 25082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:32.866768 25082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:32.866936 25082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:32.867015 25082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:32.903457 25082 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:36553
I20260811 00:19:32.903527 25296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:36553 every 8 connection(s)
I20260811 00:19:32.905224 25082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:32.914515 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:32.908155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:38705 (local address 127.16.71.190:35293)
0811 00:19:32.908355 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:32.908360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:32.910081 (+ 1721us) server_negotiation.cc:407] Connection header received
0811 00:19:32.911952 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:32.911958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:32.912056 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:32.912153 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:32.913728 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:32.913735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:32.913738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:32.913758 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:32.913794 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:32.913806 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:32.913925 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:32.914130 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 00:19:32.914211 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:19:32.915266 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 25082
I20260811 00:19:32.915472 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:32.915989 25303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:32.908787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:35293 (local address 127.16.71.130:38705)
0811 00:19:32.909871 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:32.909921 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:32.911582 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:32.912467 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:32.912477 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:32.912593 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:32.913360 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:32.913363 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:32.913406 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:32.913521 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:32.913902 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:32.913914 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:32.913922 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:19:32.914217 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 00:19:32.914559 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":320,"threads_started":1}
I20260811 00:19:32.916067 16670 external_mini_cluster.cc:1428] Setting key 9509ac0c015bc0790ca17c87626bd9a8
I20260811 00:19:32.918550 25298 heartbeater.cc:347] Connected to a master server at 127.16.71.190:35293
I20260811 00:19:32.918941 25298 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:32.919626 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:35293
--builtin_ntp_servers=127.16.71.148:45489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 00:19:32.919831 25298 heartbeater.cc:511] Master 127.16.71.190:35293 requested a full tablet report, sending...
I20260811 00:19:32.921608 24761 ts_manager.cc:194] Registered new tserver with Master: d3923f52c8594666916e5e4e304e21b5 (127.16.71.130:36553)
W20260811 00:19:33.284719 25304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:33.285046 25304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:33.285187 25304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:33.293066 25304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:33.293207 25304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:33.293356 25304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:33.303265 25304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:35293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:33.304762 25304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:33.306851 25304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:33.321527 25310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:33.321491 25311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:33.321591 25313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:33.322330 25304 server_base.cc:1034] running on GCE node
I20260811 00:19:33.323065 25304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:33.324160 25304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:33.325407 25304 hybrid_clock.cc:648] HybridClock initialized: now 1786407573325331 us; error 58 us; skew 500 ppm
I20260811 00:19:33.325775 25304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:33.328459 25304 webserver.cc:460] Webserver started at http://127.16.71.131:43265/ using document root <none> and password file <none>
I20260811 00:19:33.329326 25304 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:33.329454 25304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:33.329815 25304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:33.333321 25304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "1b00b539e0cb4010b13d97daef058b4f"
format_stamp: "Formatted at 2026-08-11 00:19:33 on dist-test-slave-f7th"
server_key: "3f1d677e1106b13486340ae5e57109a9"
server_key_iv: "7ff774acb267d7dc04d989166d9136e0"
server_key_version: "encryptionkey@0"
I20260811 00:19:33.334251 25304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "1b00b539e0cb4010b13d97daef058b4f"
format_stamp: "Formatted at 2026-08-11 00:19:33 on dist-test-slave-f7th"
server_key: "3f1d677e1106b13486340ae5e57109a9"
server_key_iv: "7ff774acb267d7dc04d989166d9136e0"
server_key_version: "encryptionkey@0"
I20260811 00:19:33.341152 25304 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:19:33.345669 25328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:33.347419 25304 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:19:33.347636 25304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "1b00b539e0cb4010b13d97daef058b4f"
format_stamp: "Formatted at 2026-08-11 00:19:33 on dist-test-slave-f7th"
server_key: "3f1d677e1106b13486340ae5e57109a9"
server_key_iv: "7ff774acb267d7dc04d989166d9136e0"
server_key_version: "encryptionkey@0"
I20260811 00:19:33.347888 25304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:33.354233 25304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:33.355154 25304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:33.355433 25304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:33.356485 25304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:33.358109 25304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:33.358220 25304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:33.358340 25304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:33.358409 25304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:33.392843 25304 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:41185
I20260811 00:19:33.392839 25523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:41185 every 8 connection(s)
I20260811 00:19:33.394410 25304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:33.397068 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 25304
I20260811 00:19:33.397289 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:33.397737 16670 external_mini_cluster.cc:1428] Setting key 15374d543b2c9b1eac1e20cfcf5b2383
I20260811 00:19:33.403870 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.397313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:39331 (local address 127.16.71.190:35293)
0811 00:19:33.397570 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:33.397576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:33.399339 (+ 1763us) server_negotiation.cc:407] Connection header received
0811 00:19:33.401611 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:33.401616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:33.401672 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:33.401758 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:33.403190 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:33.403195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:33.403199 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:33.403217 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:33.403255 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:33.403267 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:33.403375 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:33.403569 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 00:19:33.403635 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 00:19:33.405061 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.397946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:35293 (local address 127.16.71.131:39331)
0811 00:19:33.399160 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:33.399210 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:33.401315 (+ 2105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:33.401984 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:33.401992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:33.402102 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:33.402801 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:33.402805 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:33.402850 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:33.402985 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:33.403373 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:33.403382 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:33.403389 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:19:33.403646 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 00:19:33.403889 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":362,"threads_started":1}
I20260811 00:19:33.406756 25527 heartbeater.cc:347] Connected to a master server at 127.16.71.190:35293
I20260811 00:19:33.407091 25527 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:33.407740 25527 heartbeater.cc:511] Master 127.16.71.190:35293 requested a full tablet report, sending...
I20260811 00:19:33.409332 24761 ts_manager.cc:194] Registered new tserver with Master: 1b00b539e0cb4010b13d97daef058b4f (127.16.71.131:41185)
I20260811 00:19:33.413681 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:19:33.421084 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.418426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51574 (local address 127.16.71.190:35293)
0811 00:19:33.418596 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:33.418603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:33.419074 (+ 471us) server_negotiation.cc:407] Connection header received
0811 00:19:33.419256 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:33.419260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:33.419318 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:33.419405 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:33.420391 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:33.420397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:33.420402 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:33.420421 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:33.420455 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:33.420467 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:33.420582 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:33.420722 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 00:19:33.420790 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:19:33.425631 24761 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:33.432909 24761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:33.444389 25076 heartbeater.cc:503] Master 127.16.71.190:35293 was elected leader, sending a full tablet report...
I20260811 00:19:33.451828 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.446684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.16.71.131:41185)
0811 00:19:33.447529 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:33.447539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:33.448450 (+ 911us) server_negotiation.cc:407] Connection header received
0811 00:19:33.448545 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:33.448555 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:33.448871 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:33.449048 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:33.450784 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:33.450794 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:33.450812 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:33.450831 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:33.450889 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:33.450910 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:33.451116 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:33.451189 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 00:19:33.451310 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":488,"threads_started":1}
I20260811 00:19:33.451977 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.446306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:41185 (local address 127.0.0.1:39616)
0811 00:19:33.448045 (+ 1739us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:33.448108 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:33.448283 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:33.449104 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:33.449113 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:33.449215 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:33.450153 (+ 938us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:33.450156 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:33.450191 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:33.450296 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:33.451025 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:33.451036 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:33.451044 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:19:33.451228 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 00:19:33.451337 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1444,"thread_start_us":1254,"threads_started":1}
I20260811 00:19:33.456607 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.448008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:36553 (local address 127.0.0.1:58144)
0811 00:19:33.453066 (+ 5058us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:33.453084 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:33.453171 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:33.454620 (+ 1449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:33.454625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:33.454669 (+ 44us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:33.455480 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:33.455483 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:33.455496 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:33.455532 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:33.456062 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:33.456067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:33.456073 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 00:19:33.456177 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 00:19:33.456247 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4966,"thread_start_us":4290,"threads_started":1}
I20260811 00:19:33.456739 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.447375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44144)
0811 00:19:33.452589 (+ 5214us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:33.452611 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:33.452752 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:33.454620 (+ 1868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:33.454625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:33.454656 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:33.455366 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:33.455369 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:33.455392 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:33.455452 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:33.456083 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:33.456088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:33.456092 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 00:19:33.456215 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 00:19:33.456251 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5080,"thread_start_us":7730,"threads_started":1}
I20260811 00:19:33.457340 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.448238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.16.71.130:36553)
0811 00:19:33.448793 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:33.448803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:33.453297 (+ 4494us) server_negotiation.cc:407] Connection header received
0811 00:19:33.453390 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:33.453397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:33.453594 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:33.453726 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:33.455709 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:33.455719 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:33.455734 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:33.455766 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:33.455831 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:33.455853 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:33.455997 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:33.456920 (+ 923us) server_negotiation.cc:299] Negotiation successful
0811 00:19:33.457027 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":276,"threads_started":1}
I20260811 00:19:33.457386 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.446988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.16.71.129:37837)
0811 00:19:33.453017 (+ 6029us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:33.453027 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:33.453058 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:19:33.453132 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:33.453141 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:33.453393 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:33.453546 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:33.455747 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:33.455755 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:33.455765 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:33.455784 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:33.455838 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:33.455856 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:33.456035 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:33.456914 (+ 879us) server_negotiation.cc:299] Negotiation successful
0811 00:19:33.457027 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5819,"thread_start_us":5661,"threads_started":1}
I20260811 00:19:33.457000 25412 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3c5c1570b348a8b6050701a9832173 (DEFAULT_TABLE table=test-table [id=1e10df3825b241d380baccbd836cbe53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:33.460210 25412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3c5c1570b348a8b6050701a9832173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:33.461455 24961 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3c5c1570b348a8b6050701a9832173 (DEFAULT_TABLE table=test-table [id=1e10df3825b241d380baccbd836cbe53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:33.461485 25188 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3c5c1570b348a8b6050701a9832173 (DEFAULT_TABLE table=test-table [id=1e10df3825b241d380baccbd836cbe53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:33.462823 24961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3c5c1570b348a8b6050701a9832173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:33.462931 25188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3c5c1570b348a8b6050701a9832173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:33.475262 25563 tablet_bootstrap.cc:492] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Bootstrap starting.
I20260811 00:19:33.478637 25566 tablet_bootstrap.cc:492] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Bootstrap starting.
I20260811 00:19:33.479873 25565 tablet_bootstrap.cc:492] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Bootstrap starting.
I20260811 00:19:33.481750 25563 tablet_bootstrap.cc:654] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:33.482959 25565 tablet_bootstrap.cc:654] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:33.483136 25566 tablet_bootstrap.cc:654] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:33.483239 25563 log.cc:824] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:33.484369 25565 log.cc:824] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:33.484769 25566 log.cc:824] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:33.488215 25565 tablet_bootstrap.cc:492] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: No bootstrap required, opened a new log
I20260811 00:19:33.488233 25566 tablet_bootstrap.cc:492] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: No bootstrap required, opened a new log
I20260811 00:19:33.488540 25565 ts_tablet_manager.cc:1397] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 00:19:33.488560 25566 ts_tablet_manager.cc:1397] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 00:19:33.489295 25563 tablet_bootstrap.cc:492] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: No bootstrap required, opened a new log
I20260811 00:19:33.489629 25563 ts_tablet_manager.cc:1397] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.004s
I20260811 00:19:33.496784 25565 raft_consensus.cc:348] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.497225 25565 raft_consensus.cc:374] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:33.497339 25565 raft_consensus.cc:729] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b00b539e0cb4010b13d97daef058b4f, State: Initialized, Role: FOLLOWER
I20260811 00:19:33.497992 25565 consensus_queue.cc:260] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.499167 25527 heartbeater.cc:503] Master 127.16.71.190:35293 was elected leader, sending a full tablet report...
I20260811 00:19:33.499502 25566 raft_consensus.cc:348] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.499927 25565 ts_tablet_manager.cc:1428] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260811 00:19:33.499958 25566 raft_consensus.cc:374] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:33.500058 25566 raft_consensus.cc:729] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3923f52c8594666916e5e4e304e21b5, State: Initialized, Role: FOLLOWER
I20260811 00:19:33.500399 25563 raft_consensus.cc:348] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.500916 25563 raft_consensus.cc:374] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:33.500777 25566 consensus_queue.cc:260] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.501022 25563 raft_consensus.cc:729] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5b71001ff04c9ab7abc6050942c441, State: Initialized, Role: FOLLOWER
I20260811 00:19:33.501678 25563 consensus_queue.cc:260] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.503073 25566 ts_tablet_manager.cc:1428] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 00:19:33.503921 25298 heartbeater.cc:503] Master 127.16.71.190:35293 was elected leader, sending a full tablet report...
I20260811 00:19:33.507597 25563 ts_tablet_manager.cc:1428] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Time spent starting tablet: real 0.018s user 0.009s sys 0.008s
W20260811 00:19:33.646242 25529 tablet.cc:2369] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:19:33.657433 25299 tablet.cc:2369] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:19:33.678064 25077 tablet.cc:2369] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:33.918352 25575 raft_consensus.cc:482] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:33.918710 25575 raft_consensus.cc:504] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.920670 25575 leader_election.cc:288] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b00b539e0cb4010b13d97daef058b4f (127.16.71.131:41185), d3923f52c8594666916e5e4e304e21b5 (127.16.71.130:36553)
I20260811 00:19:33.924175 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.921189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:46615 (local address 127.16.71.131:41185)
0811 00:19:33.921436 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:33.921443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:33.922129 (+ 686us) server_negotiation.cc:407] Connection header received
0811 00:19:33.922318 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:33.922323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:33.922382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:33.922467 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:33.923530 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:33.923536 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:33.923540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:33.923557 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:33.923592 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:33.923604 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:33.923720 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:33.923805 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 00:19:33.923877 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 00:19:33.924369 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.921338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:41185 (local address 127.16.71.129:46615)
0811 00:19:33.921988 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:33.922005 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:33.922176 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:33.922541 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:33.922546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:33.922571 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:33.923287 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:33.923290 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:33.923310 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:33.923364 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:33.923697 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:33.923702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:33.923707 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 00:19:33.923843 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 00:19:33.923894 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":655,"threads_started":1}
I20260811 00:19:33.924507 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.921174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:54335 (local address 127.16.71.130:36553)
0811 00:19:33.921444 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:33.921451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:33.922576 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 00:19:33.922726 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:33.922730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:33.922780 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:33.922860 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:33.923804 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:33.923809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:33.923812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:33.923828 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:33.923903 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:33.923916 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:33.924042 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:33.924162 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 00:19:33.924226 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 00:19:33.924746 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:33.921804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:36553 (local address 127.16.71.129:54335)
0811 00:19:33.922454 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:33.922468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:33.922589 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:33.922918 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:33.922923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:33.922946 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:33.923574 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:19:33.923577 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:33.923590 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:19:33.923629 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:33.924041 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:33.924046 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:33.924051 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 00:19:33.924218 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 00:19:33.924273 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":425,"threads_started":1}
I20260811 00:19:33.925547 25446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c5c1570b348a8b6050701a9832173" candidate_uuid: "3f5b71001ff04c9ab7abc6050942c441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b00b539e0cb4010b13d97daef058b4f" is_pre_election: true
I20260811 00:19:33.925737 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c5c1570b348a8b6050701a9832173" candidate_uuid: "3f5b71001ff04c9ab7abc6050942c441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3923f52c8594666916e5e4e304e21b5" is_pre_election: true
I20260811 00:19:33.926035 25446 raft_consensus.cc:2454] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5b71001ff04c9ab7abc6050942c441 in term 0.
I20260811 00:19:33.926331 25220 raft_consensus.cc:2454] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5b71001ff04c9ab7abc6050942c441 in term 0.
I20260811 00:19:33.926748 24881 leader_election.cc:302] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b00b539e0cb4010b13d97daef058b4f, 3f5b71001ff04c9ab7abc6050942c441; no voters:
I20260811 00:19:33.927259 25575 raft_consensus.cc:2781] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:33.927414 25575 raft_consensus.cc:482] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:33.927520 25575 raft_consensus.cc:3037] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:33.929922 25575 raft_consensus.cc:504] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.930780 25575 leader_election.cc:288] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [CANDIDATE]: Term 1 election: Requested vote from peers 1b00b539e0cb4010b13d97daef058b4f (127.16.71.131:41185), d3923f52c8594666916e5e4e304e21b5 (127.16.71.130:36553)
I20260811 00:19:33.931382 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c5c1570b348a8b6050701a9832173" candidate_uuid: "3f5b71001ff04c9ab7abc6050942c441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3923f52c8594666916e5e4e304e21b5"
I20260811 00:19:33.931387 25446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c5c1570b348a8b6050701a9832173" candidate_uuid: "3f5b71001ff04c9ab7abc6050942c441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b00b539e0cb4010b13d97daef058b4f"
I20260811 00:19:33.931591 25446 raft_consensus.cc:3037] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:33.931596 25220 raft_consensus.cc:3037] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:33.934005 25446 raft_consensus.cc:2454] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5b71001ff04c9ab7abc6050942c441 in term 1.
I20260811 00:19:33.934383 25220 raft_consensus.cc:2454] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5b71001ff04c9ab7abc6050942c441 in term 1.
I20260811 00:19:33.934626 24881 leader_election.cc:302] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b00b539e0cb4010b13d97daef058b4f, 3f5b71001ff04c9ab7abc6050942c441; no voters:
I20260811 00:19:33.934973 25575 raft_consensus.cc:2781] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:33.935634 25575 raft_consensus.cc:686] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 1 LEADER]: Becoming Leader. State: Replica: 3f5b71001ff04c9ab7abc6050942c441, State: Running, Role: LEADER
I20260811 00:19:33.936141 25575 consensus_queue.cc:237] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } }
I20260811 00:19:33.942065 24761 catalog_manager.cc:5495] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f5b71001ff04c9ab7abc6050942c441 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "3f5b71001ff04c9ab7abc6050942c441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5b71001ff04c9ab7abc6050942c441" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:34.034413 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.031441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.16.71.129:37837)
0811 00:19:34.032351 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:34.032358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:34.032377 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:19:34.032479 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:34.032483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:34.032568 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:34.032712 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:34.033753 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:34.033757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:34.033761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:34.033779 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:34.033823 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:34.033835 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:34.033958 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:34.034067 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 00:19:34.034131 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":645,"threads_started":1}
I20260811 00:19:34.043171 25446 raft_consensus.cc:1261] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 1 FOLLOWER]: Refusing update from remote peer 3f5b71001ff04c9ab7abc6050942c441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:34.043200 25220 raft_consensus.cc:1261] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f5b71001ff04c9ab7abc6050942c441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:34.044248 25578 consensus_queue.cc:1035] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3923f52c8594666916e5e4e304e21b5" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:34.044641 25575 consensus_queue.cc:1035] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b00b539e0cb4010b13d97daef058b4f" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:34.073030 24761 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:34.073365 24761 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:34.077849 24761 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e662ab40f214793879ed9a420189049: Sending DeleteTablet for 3 replicas of tablet 2e3c5c1570b348a8b6050701a9832173
I20260811 00:19:34.079344 25412 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3c5c1570b348a8b6050701a9832173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:34 UTC) from {username='slave'} at 127.0.0.1:39616
I20260811 00:19:34.079355 24961 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3c5c1570b348a8b6050701a9832173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:34 UTC) from {username='slave'} at 127.0.0.1:44144
I20260811 00:19:34.079679 25188 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3c5c1570b348a8b6050701a9832173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:34 UTC) from {username='slave'} at 127.0.0.1:58144
I20260811 00:19:34.082166 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24854
I20260811 00:19:34.090430 25627 tablet_replica.cc:330] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: stopping tablet replica
I20260811 00:19:34.091491 25626 tablet_replica.cc:330] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: stopping tablet replica
I20260811 00:19:34.091907 25625 tablet_replica.cc:330] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: stopping tablet replica
I20260811 00:19:34.091902 25627 raft_consensus.cc:2229] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:34.091974 25626 raft_consensus.cc:2229] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:34.092468 25625 raft_consensus.cc:2229] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:34.092548 25626 raft_consensus.cc:2258] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:34.092571 25627 raft_consensus.cc:2258] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:34.093922 25625 raft_consensus.cc:2258] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:34.094969 25626 ts_tablet_manager.cc:1905] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:34.094966 25627 ts_tablet_manager.cc:1905] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:34.096058 25625 ts_tablet_manager.cc:1905] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:34.099833 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.083706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.16.71.129:37837)
0811 00:19:34.083926 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:34.083932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:34.097059 (+ 13127us) server_negotiation.cc:407] Connection header received
0811 00:19:34.097572 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:34.097577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:34.097649 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:34.097736 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:34.098945 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:34.098951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:34.098956 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:34.098992 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:34.099034 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:34.099051 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:34.099147 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:34.099406 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 00:19:34.099477 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:19:34.101202 25068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:34.102612 25627 ts_tablet_manager.cc:1918] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:34.102607 25626 ts_tablet_manager.cc:1918] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:34.102825 25626 log.cc:1196] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/2e3c5c1570b348a8b6050701a9832173
I20260811 00:19:34.102906 25627 log.cc:1196] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/2e3c5c1570b348a8b6050701a9832173
I20260811 00:19:34.103695 25627 ts_tablet_manager.cc:1939] T 2e3c5c1570b348a8b6050701a9832173 P d3923f52c8594666916e5e4e304e21b5: Deleting consensus metadata
I20260811 00:19:34.103747 25626 ts_tablet_manager.cc:1939] T 2e3c5c1570b348a8b6050701a9832173 P 1b00b539e0cb4010b13d97daef058b4f: Deleting consensus metadata
I20260811 00:19:34.105685 24737 catalog_manager.cc:4841] TS 1b00b539e0cb4010b13d97daef058b4f (127.16.71.131:41185): tablet 2e3c5c1570b348a8b6050701a9832173 (table test-table [id=1e10df3825b241d380baccbd836cbe53]) successfully deleted
I20260811 00:19:34.105890 24740 catalog_manager.cc:4841] TS d3923f52c8594666916e5e4e304e21b5 (127.16.71.130:36553): tablet 2e3c5c1570b348a8b6050701a9832173 (table test-table [id=1e10df3825b241d380baccbd836cbe53]) successfully deleted
I20260811 00:19:34.200536 25625 ts_tablet_manager.cc:1918] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:34.200809 25625 log.cc:1196] T 2e3c5c1570b348a8b6050701a9832173 P 3f5b71001ff04c9ab7abc6050942c441: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/2e3c5c1570b348a8b6050701a9832173
I20260811 00:19:34.200855 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24854
I20260811 00:19:34.226046 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 25082
W20260811 00:19:34.225979 24739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.71.129:37837 (error 108)
I20260811 00:19:34.228376 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.227556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44180)
0811 00:19:34.228020 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:34.228162 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":201,"threads_started":1}
W20260811 00:19:34.228691 24739 catalog_manager.cc:4584] TS 3f5b71001ff04c9ab7abc6050942c441 (127.16.71.129:37837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e3c5c1570b348a8b6050701a9832173: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
I20260811 00:19:34.229301 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.226846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.16.71.130:36553)
0811 00:19:34.227288 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:34.227294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:34.227317 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:19:34.227388 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:34.227394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:34.227484 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:34.227570 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:34.228702 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:34.228707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:34.228710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:34.228726 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:34.228758 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:34.228770 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:34.228880 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:34.229028 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 00:19:34.229087 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
I20260811 00:19:34.229981 25290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:34.282598 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.282044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44184)
0811 00:19:34.282338 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:34.282450 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 00:19:34.283973 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.283505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44190)
0811 00:19:34.283680 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:34.283784 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 00:19:34.321296 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 25082
I20260811 00:19:34.344901 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.344317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44206)
0811 00:19:34.344582 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:34.344707 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 00:19:34.346052 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.345620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44222)
0811 00:19:34.345778 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:34.345889 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 00:19:34.350965 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 25304
I20260811 00:19:34.354043 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.351927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39618 (local address 127.16.71.131:41185)
0811 00:19:34.352110 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:34.352116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:34.352137 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:19:34.352203 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:34.352208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:34.352304 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:34.352444 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:34.353402 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:34.353407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:34.353411 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:34.353428 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:34.353461 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:34.353474 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:34.353597 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:34.353681 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 00:19:34.353745 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:19:34.354740 25515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:34.426975 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.426396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44238)
0811 00:19:34.426679 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:34.426785 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 00:19:34.428190 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:34.427743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37837 (local address 127.0.0.1:44254)
0811 00:19:34.427957 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:34.428041 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 00:19:34.444247 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 25304
I20260811 00:19:34.473831 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24706
I20260811 00:19:34.475278 24808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:34.585127 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24706
2026-08-11T00:19:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3303 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 00:19:34.636407 16670 mini_oidc.cc:152] Starting JWKS server
I20260811 00:19:34.639489 16670 webserver.cc:460] Webserver started at https://127.0.0.1:43385/ using document root <none> and password file <none>
I20260811 00:19:34.639853 16670 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:19:34.640801 16670 webserver.cc:460] Webserver started at http://127.0.0.1:35427/ using document root <none> and password file <none>
2026-08-11T00:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:34Z Disabled control of system clock
I20260811 00:19:34.678274 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34207
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:34207
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43385/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/testchainca.cert with env {}
W20260811 00:19:35.046223 25648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:35.046568 25648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:35.046659 25648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:35.054244 25648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:35.054375 25648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:35.054414 25648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:35.054463 25648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:35.054509 25648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:19:35.054576 25648 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:19:35.054615 25648 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43385/jwks.json
I20260811 00:19:35.065482 25648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:34207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43385/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:35.066864 25648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:35.068228 25648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:35.080812 25661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:35.080775 25662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:35.080840 25664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:35.081597 25648 server_base.cc:1034] running on GCE node
I20260811 00:19:35.082470 25648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:35.084084 25648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:35.085309 25648 hybrid_clock.cc:648] HybridClock initialized: now 1786407575085254 us; error 46 us; skew 500 ppm
I20260811 00:19:35.085685 25648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:35.088183 25648 webserver.cc:460] Webserver started at http://127.16.71.190:34333/ using document root <none> and password file <none>
I20260811 00:19:35.088959 25648 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:35.089093 25648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:35.089409 25648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:35.092831 25648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "aed44610fac54708ae27b537833cbd18"
format_stamp: "Formatted at 2026-08-11 00:19:35 on dist-test-slave-f7th"
server_key: "c1ccaf050bc618f81056410d36b9b0cc"
server_key_iv: "5ac9fc436a484e3d5cb047e109d419f7"
server_key_version: "encryptionkey@0"
I20260811 00:19:35.093755 25648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "aed44610fac54708ae27b537833cbd18"
format_stamp: "Formatted at 2026-08-11 00:19:35 on dist-test-slave-f7th"
server_key: "c1ccaf050bc618f81056410d36b9b0cc"
server_key_iv: "5ac9fc436a484e3d5cb047e109d419f7"
server_key_version: "encryptionkey@0"
I20260811 00:19:35.099735 25648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 00:19:35.103900 25674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:35.105588 25648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:19:35.105803 25648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "aed44610fac54708ae27b537833cbd18"
format_stamp: "Formatted at 2026-08-11 00:19:35 on dist-test-slave-f7th"
server_key: "c1ccaf050bc618f81056410d36b9b0cc"
server_key_iv: "5ac9fc436a484e3d5cb047e109d419f7"
server_key_version: "encryptionkey@0"
I20260811 00:19:35.106106 25648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:35.140594 25648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:35.141556 25648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:35.141892 25648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:35.160838 25648 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:34207
I20260811 00:19:35.160859 25755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:34207 every 8 connection(s)
I20260811 00:19:35.162304 25648 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:35.165938 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 25648
I20260811 00:19:35.166132 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:35.166622 25761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:35.166805 16670 external_mini_cluster.cc:1428] Setting key ebe6852f21ec32d23a7c6b271c939ae6
I20260811 00:19:35.183856 25761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18: Bootstrap starting.
I20260811 00:19:35.184533 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:35.169117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.16.71.190:34207)
0811 00:19:35.170363 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:35.170393 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:35.170471 (+ 78us) server_negotiation.cc:407] Connection header received
0811 00:19:35.172566 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:35.172605 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:35.173421 (+ 816us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:35.173980 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:35.174701 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:35.176820 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:35.178390 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:35.179484 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:35.179710 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:35.181244 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:35.181258 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:35.181283 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:35.182087 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:35.182208 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:35.182242 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:35.182378 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:35.182678 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 00:19:35.183067 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":385,"threads_started":1}
I20260811 00:19:35.187731 25761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:35.189119 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:19:35.189087 25761 log.cc:824] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:35.192395 25761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18: No bootstrap required, opened a new log
I20260811 00:19:35.200984 25761 raft_consensus.cc:348] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed44610fac54708ae27b537833cbd18" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34207 } }
I20260811 00:19:35.201465 25761 raft_consensus.cc:374] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:35.201583 25761 raft_consensus.cc:729] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed44610fac54708ae27b537833cbd18, State: Initialized, Role: FOLLOWER
I20260811 00:19:35.202313 25761 consensus_queue.cc:260] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed44610fac54708ae27b537833cbd18" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34207 } }
I20260811 00:19:35.202638 25761 raft_consensus.cc:388] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:35.202800 25761 raft_consensus.cc:482] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:35.203344 25761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:35.206002 25761 raft_consensus.cc:504] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed44610fac54708ae27b537833cbd18" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34207 } }
I20260811 00:19:35.206655 25761 leader_election.cc:302] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aed44610fac54708ae27b537833cbd18; no voters:
I20260811 00:19:35.207350 25761 leader_election.cc:288] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:35.207499 25779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:35.208202 25779 raft_consensus.cc:686] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [term 1 LEADER]: Becoming Leader. State: Replica: aed44610fac54708ae27b537833cbd18, State: Running, Role: LEADER
I20260811 00:19:35.208905 25779 consensus_queue.cc:237] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed44610fac54708ae27b537833cbd18" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34207 } }
I20260811 00:19:35.209666 25761 sys_catalog.cc:564] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:35.212805 25784 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aed44610fac54708ae27b537833cbd18. Latest consensus state: current_term: 1 leader_uuid: "aed44610fac54708ae27b537833cbd18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed44610fac54708ae27b537833cbd18" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34207 } } }
I20260811 00:19:35.212733 25783 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aed44610fac54708ae27b537833cbd18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed44610fac54708ae27b537833cbd18" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34207 } } }
I20260811 00:19:35.213254 25784 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:35.213254 25783 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:35.217931 25788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:35.224818 25788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:35.235598 25788 catalog_manager.cc:1347] Generated new cluster ID: fa7af31b3abd4fddb4d158b82925a212
I20260811 00:19:35.235746 25788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:35.266492 25788 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:35.268486 25788 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:35.288220 25788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aed44610fac54708ae27b537833cbd18: Generated new TSK 0
I20260811 00:19:35.289526 25788 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 00:19:35.337677 25811 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 00:19:35.337931 25811 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:19:35.339362 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:35.314989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.16.71.190:34207)
0811 00:19:35.315205 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:35.315211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:35.315708 (+ 497us) server_negotiation.cc:407] Connection header received
0811 00:19:35.316108 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:35.316115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:35.316217 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:35.316404 (+ 187us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:19:35.317038 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:35.318253 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:35.319916 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:35.320703 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:35.320860 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:35.321617 (+ 757us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:19:35.338569 (+ 16952us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 00:19:35.339006 (+ 437us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54938: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 00:19:35.339517 25769 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54938: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 00:19:35.341187 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:35.314891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34207 (local address 127.0.0.1:54938)
0811 00:19:35.315592 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:35.315610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:35.315778 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:35.316474 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:35.316478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:35.316489 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:19:35.316876 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:35.316887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:35.318446 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:35.318451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:35.319728 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:35.319744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:35.320952 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:35.320958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:35.321115 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:35.321201 (+ 86us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:19:35.338974 (+ 17773us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 00:19:35.339312 (+ 338us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.71.190:34207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":355,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.16.71.190:34207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 00:19:35.343055 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 25648
I20260811 00:19:35.344370 25749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:35.446349 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 25648
2026-08-11T00:19:35Z chronyd exiting
I20260811 00:19:35.505993 16670 test_util.cc:175] -----------------------------------------------
I20260811 00:19:35.506132 16670 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407527335556-16670-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (874 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 00:19:35.510114 16670 mini_oidc.cc:152] Starting JWKS server
I20260811 00:19:35.512156 16670 webserver.cc:460] Webserver started at https://127.0.0.1:35575/ using document root <none> and password file <none>
I20260811 00:19:35.512394 16670 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:19:35.513293 16670 webserver.cc:460] Webserver started at http://127.0.0.1:34779/ using document root <none> and password file <none>
2026-08-11T00:19:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:35Z Disabled control of system clock
I20260811 00:19:35.550721 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:35227
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:35227
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35575/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/testchainca.cert with env {}
W20260811 00:19:35.915509 25831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:35.915863 25831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:35.915982 25831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:35.923592 25831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:35.923740 25831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:35.923779 25831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:35.923866 25831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:35.923924 25831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:19:35.923983 25831 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:19:35.924160 25831 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35575/jwks.json
I20260811 00:19:35.935667 25831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:35227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:35227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35575/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:35.938082 25831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:35.939742 25831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:35.953802 25841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:35.953990 25846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:35.953958 25842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:35.954604 25831 server_base.cc:1034] running on GCE node
I20260811 00:19:35.955765 25831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:35.957446 25831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:35.958760 25831 hybrid_clock.cc:648] HybridClock initialized: now 1786407575958702 us; error 50 us; skew 500 ppm
I20260811 00:19:35.959216 25831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:35.962148 25831 webserver.cc:460] Webserver started at http://127.16.71.190:36873/ using document root <none> and password file <none>
I20260811 00:19:35.962955 25831 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:35.963084 25831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:35.963435 25831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:35.966965 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "0cd5ed417a40462c81d8f8a4c955bdb0"
format_stamp: "Formatted at 2026-08-11 00:19:35 on dist-test-slave-f7th"
server_key: "eea044d0a60bf8a08a35da0f0707030b"
server_key_iv: "cbc7c54acd6e244d5b0ad1618b896016"
server_key_version: "encryptionkey@0"
I20260811 00:19:35.968065 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "0cd5ed417a40462c81d8f8a4c955bdb0"
format_stamp: "Formatted at 2026-08-11 00:19:35 on dist-test-slave-f7th"
server_key: "eea044d0a60bf8a08a35da0f0707030b"
server_key_iv: "cbc7c54acd6e244d5b0ad1618b896016"
server_key_version: "encryptionkey@0"
I20260811 00:19:35.975783 25831 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 00:19:35.980563 25853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:35.982316 25831 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:35.982545 25831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "0cd5ed417a40462c81d8f8a4c955bdb0"
format_stamp: "Formatted at 2026-08-11 00:19:35 on dist-test-slave-f7th"
server_key: "eea044d0a60bf8a08a35da0f0707030b"
server_key_iv: "cbc7c54acd6e244d5b0ad1618b896016"
server_key_version: "encryptionkey@0"
I20260811 00:19:35.982807 25831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:36.010833 25831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:36.011838 25831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:36.012207 25831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:36.033368 25831 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:35227
I20260811 00:19:36.033388 25944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:35227 every 8 connection(s)
I20260811 00:19:36.034921 25831 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:36.037755 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 25831
I20260811 00:19:36.037998 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:36.038810 16670 external_mini_cluster.cc:1428] Setting key c48a6efa8c21d28aa01ff0252d2d2921
I20260811 00:19:36.039820 25945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:36.053670 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:36.041010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.16.71.190:35227)
0811 00:19:36.042196 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:36.042209 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:36.042256 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:19:36.043849 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:36.043875 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:36.044437 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:36.044916 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:36.045526 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.046971 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.048263 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.049030 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.049216 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:36.050397 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:36.050406 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:36.050423 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:36.051049 (+ 626us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:36.051132 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:36.051155 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:36.051289 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:36.051515 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 00:19:36.051854 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":405,"threads_started":1}
I20260811 00:19:36.053941 25945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0: Bootstrap starting.
I20260811 00:19:36.057811 25945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:36.058269 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:19:36.059208 25945 log.cc:824] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:36.062739 25945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0: No bootstrap required, opened a new log
I20260811 00:19:36.071019 25945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35227 } }
I20260811 00:19:36.071463 25945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:36.071544 25945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd5ed417a40462c81d8f8a4c955bdb0, State: Initialized, Role: FOLLOWER
I20260811 00:19:36.072118 25945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35227 } }
I20260811 00:19:36.072335 25945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:36.072443 25945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:36.072577 25945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:36.075538 25945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35227 } }
I20260811 00:19:36.076085 25945 leader_election.cc:302] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cd5ed417a40462c81d8f8a4c955bdb0; no voters:
I20260811 00:19:36.076694 25945 leader_election.cc:288] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:36.076848 25955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:36.077575 25955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [term 1 LEADER]: Becoming Leader. State: Replica: 0cd5ed417a40462c81d8f8a4c955bdb0, State: Running, Role: LEADER
I20260811 00:19:36.078137 25955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35227 } }
I20260811 00:19:36.078831 25945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:36.080919 25959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cd5ed417a40462c81d8f8a4c955bdb0. Latest consensus state: current_term: 1 leader_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35227 } } }
I20260811 00:19:36.081317 25959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:36.082901 25956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd5ed417a40462c81d8f8a4c955bdb0" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35227 } } }
I20260811 00:19:36.083325 25956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:36.084075 25964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:36.090391 25964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:36.101830 25964 catalog_manager.cc:1347] Generated new cluster ID: 7775c1450b1242f4bd9477191cd8ca5f
I20260811 00:19:36.101989 25964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:36.114256 25964 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:36.116537 25964 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:36.128876 25964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cd5ed417a40462c81d8f8a4c955bdb0: Generated new TSK 0
I20260811 00:19:36.130242 25964 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 00:19:36.175683 25994 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 00:19:36.175812 25994 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:19:36.177343 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:36.155511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.16.71.190:35227)
0811 00:19:36.155735 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:36.155742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:36.156218 (+ 476us) server_negotiation.cc:407] Connection header received
0811 00:19:36.156465 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:36.156472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:36.156571 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:36.156787 (+ 216us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:19:36.157369 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.158562 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.160172 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.160933 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.161093 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:36.161695 (+ 602us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:19:36.176465 (+ 14770us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 00:19:36.177004 (+ 539us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37580: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 00:19:36.177450 25993 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:36.155337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:35227 (local address 127.0.0.1:37580)
0811 00:19:36.156105 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:36.156123 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:36.156266 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:36.156791 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:36.156795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:36.156805 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:19:36.157156 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:36.157169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.158795 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.158800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:36.159990 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:36.160005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.161145 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.161150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:36.161311 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:36.161355 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:19:36.176819 (+ 15464us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 00:19:36.177112 (+ 293us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.71.190:35227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":421,"threads_started":1}
W20260811 00:19:36.177505 25953 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37580: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 00:19:36.178822 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 25831
I20260811 00:19:36.180384 25937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:36.276548 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 25831
2026-08-11T00:19:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (832 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 00:19:36.343036 16670 mini_oidc.cc:152] Starting JWKS server
I20260811 00:19:36.345217 16670 webserver.cc:460] Webserver started at https://127.0.0.1:45597/ using document root <none> and password file <none>
I20260811 00:19:36.345494 16670 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:19:36.346297 16670 webserver.cc:460] Webserver started at http://127.0.0.1:37059/ using document root <none> and password file <none>
2026-08-11T00:19:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:36Z Disabled control of system clock
I20260811 00:19:36.384469 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40945
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:35735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:40945
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45597/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 00:19:36.749121 26005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:36.749465 26005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:36.749606 26005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:36.757336 26005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:36.757485 26005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:36.757556 26005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:36.757639 26005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:36.757704 26005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:19:36.757791 26005 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:19:36.757843 26005 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45597/jwks.json
I20260811 00:19:36.768113 26005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:35735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:40945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45597/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:36.769870 26005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:36.772379 26005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:36.785838 26012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:36.785811 26014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:36.785759 26017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:36.787329 26005 server_base.cc:1034] running on GCE node
I20260811 00:19:36.788360 26005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:36.790278 26005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:36.791538 26005 hybrid_clock.cc:648] HybridClock initialized: now 1786407576791500 us; error 39 us; skew 500 ppm
I20260811 00:19:36.791978 26005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:36.798566 26005 webserver.cc:460] Webserver started at http://127.16.71.190:41387/ using document root <none> and password file <none>
I20260811 00:19:36.799468 26005 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:36.799623 26005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:36.800020 26005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:36.803639 26005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "1f1b4ec4f25f44248b864b8e3d533d79"
format_stamp: "Formatted at 2026-08-11 00:19:36 on dist-test-slave-f7th"
server_key: "c11603ccc49ce5dcf6b28e868dcd46a0"
server_key_iv: "503bf5f5cca116d2b750e424b7031692"
server_key_version: "encryptionkey@0"
I20260811 00:19:36.804554 26005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "1f1b4ec4f25f44248b864b8e3d533d79"
format_stamp: "Formatted at 2026-08-11 00:19:36 on dist-test-slave-f7th"
server_key: "c11603ccc49ce5dcf6b28e868dcd46a0"
server_key_iv: "503bf5f5cca116d2b750e424b7031692"
server_key_version: "encryptionkey@0"
I20260811 00:19:36.810690 26005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 00:19:36.814996 26026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:36.816993 26005 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:36.817219 26005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "1f1b4ec4f25f44248b864b8e3d533d79"
format_stamp: "Formatted at 2026-08-11 00:19:36 on dist-test-slave-f7th"
server_key: "c11603ccc49ce5dcf6b28e868dcd46a0"
server_key_iv: "503bf5f5cca116d2b750e424b7031692"
server_key_version: "encryptionkey@0"
I20260811 00:19:36.817445 26005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:36.839717 26005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:36.840842 26005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:36.841183 26005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:36.861428 26005 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:40945
I20260811 00:19:36.861428 26111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:40945 every 8 connection(s)
I20260811 00:19:36.862984 26005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:36.867730 26112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:36.871645 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 26005
I20260811 00:19:36.871840 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:36.872545 16670 external_mini_cluster.cc:1428] Setting key eb3c29e6eeb6cff6dc98a4aca7e76c8a
I20260811 00:19:36.882120 26112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79: Bootstrap starting.
I20260811 00:19:36.886279 26112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:36.887830 26112 log.cc:824] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:36.888922 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:36.874980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.16.71.190:40945)
0811 00:19:36.876260 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:36.876273 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:36.876327 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:19:36.878172 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:36.878201 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:36.878804 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:36.879262 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:36.879997 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.881409 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.882959 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.883753 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.884024 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:36.885491 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:36.885504 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:36.885527 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:36.886310 (+ 783us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:36.886471 (+ 161us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:36.886503 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:36.886635 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:36.886968 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 00:19:36.887652 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":477,"threads_started":1}
I20260811 00:19:36.891633 26112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79: No bootstrap required, opened a new log
I20260811 00:19:36.893956 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:19:36.901921 26112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40945 } }
I20260811 00:19:36.902503 26112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:36.902629 26112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1b4ec4f25f44248b864b8e3d533d79, State: Initialized, Role: FOLLOWER
I20260811 00:19:36.903249 26112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40945 } }
I20260811 00:19:36.903512 26112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:36.903649 26112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:36.903815 26112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:36.906301 26112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40945 } }
I20260811 00:19:36.906999 26112 leader_election.cc:302] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f1b4ec4f25f44248b864b8e3d533d79; no voters:
I20260811 00:19:36.907765 26112 leader_election.cc:288] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:36.907913 26117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:36.908596 26117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [term 1 LEADER]: Becoming Leader. State: Replica: 1f1b4ec4f25f44248b864b8e3d533d79, State: Running, Role: LEADER
I20260811 00:19:36.909262 26117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40945 } }
I20260811 00:19:36.909951 26112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:36.912089 26119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40945 } } }
I20260811 00:19:36.912257 26120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1b4ec4f25f44248b864b8e3d533d79. Latest consensus state: current_term: 1 leader_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1b4ec4f25f44248b864b8e3d533d79" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40945 } } }
I20260811 00:19:36.912523 26119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:36.912534 26120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:36.914054 26126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:36.920847 26126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:36.930491 26126 catalog_manager.cc:1347] Generated new cluster ID: 870834a19bfe468db755aba9ed95f073
I20260811 00:19:36.930655 26126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:36.942747 26126 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:36.944229 26126 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:36.953806 26126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f1b4ec4f25f44248b864b8e3d533d79: Generated new TSK 0
I20260811 00:19:36.954841 26126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:19:36.983069 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:36.972297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47686 (local address 127.16.71.190:40945)
0811 00:19:36.972669 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:36.972677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:36.973056 (+ 379us) server_negotiation.cc:407] Connection header received
0811 00:19:36.973362 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:36.973369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:36.973481 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:36.973729 (+ 248us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:19:36.974381 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.975568 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.977276 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.978031 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.978185 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:36.979087 (+ 902us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:19:36.982211 (+ 3124us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 00:19:36.982734 (+ 523us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47686: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 00:19:36.983233 26115 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47686: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 00:19:36.984738 26153 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:36.972070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40945 (local address 127.0.0.1:47686)
0811 00:19:36.972905 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:36.972933 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:36.973086 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:36.973763 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:36.973768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:36.973781 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:19:36.974178 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:36.974188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.975793 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.975798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:36.977078 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:36.977097 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:36.978307 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:36.978313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:36.978486 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:36.978545 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:19:36.982701 (+ 4156us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 00:19:36.984260 (+ 1559us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.71.190:40945: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":1168,"thread_start_us":485,"threads_started":1}
I20260811 00:19:36.986320 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 26005
I20260811 00:19:36.988494 26106 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 00:19:36.995739 26154 webserver.cc:533] Webserver: sslize failed:
E20260811 00:19:36.995885 26154 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:19:37.084343 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 26005
2026-08-11T00:19:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (802 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T00:19:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:37Z Disabled control of system clock
I20260811 00:19:37.181608 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39155
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:39155
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/testchainca.cert with env {}
W20260811 00:19:37.541052 26162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:37.541375 26162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:37.541477 26162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:37.549237 26162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:37.549366 26162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:37.549409 26162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:37.549466 26162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:37.549510 26162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:19:37.549571 26162 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:19:37.549610 26162 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 00:19:37.560420 26162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:39155
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:37.561866 26162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:37.563306 26162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:37.577267 26170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:37.577543 26171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:37.577643 26173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:37.579041 26162 server_base.cc:1034] running on GCE node
I20260811 00:19:37.579968 26162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:37.581677 26162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:37.583000 26162 hybrid_clock.cc:648] HybridClock initialized: now 1786407577582949 us; error 43 us; skew 500 ppm
I20260811 00:19:37.583400 26162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:37.586125 26162 webserver.cc:460] Webserver started at http://127.16.71.190:33731/ using document root <none> and password file <none>
I20260811 00:19:37.586966 26162 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:37.587083 26162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:37.587415 26162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:37.590951 26162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6"
format_stamp: "Formatted at 2026-08-11 00:19:37 on dist-test-slave-f7th"
server_key: "ad07931eef9f3bd4f6d3a5b3cc46557e"
server_key_iv: "388093de360343a8b28f543a376ef349"
server_key_version: "encryptionkey@0"
I20260811 00:19:37.592198 26162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6"
format_stamp: "Formatted at 2026-08-11 00:19:37 on dist-test-slave-f7th"
server_key: "ad07931eef9f3bd4f6d3a5b3cc46557e"
server_key_iv: "388093de360343a8b28f543a376ef349"
server_key_version: "encryptionkey@0"
I20260811 00:19:37.598392 26162 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 00:19:37.603255 26183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:37.604962 26162 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:19:37.605162 26162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6"
format_stamp: "Formatted at 2026-08-11 00:19:37 on dist-test-slave-f7th"
server_key: "ad07931eef9f3bd4f6d3a5b3cc46557e"
server_key_iv: "388093de360343a8b28f543a376ef349"
server_key_version: "encryptionkey@0"
I20260811 00:19:37.605422 26162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:37.629472 26162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:37.630635 26162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:37.631206 26162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:37.652340 26162 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:39155
I20260811 00:19:37.652348 26283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:39155 every 8 connection(s)
I20260811 00:19:37.653894 26162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:37.658353 26284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:37.658773 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 26162
I20260811 00:19:37.658980 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:37.659627 16670 external_mini_cluster.cc:1428] Setting key 872db934c5b511fedcf98f99e66c7f54
I20260811 00:19:37.675714 26284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6: Bootstrap starting.
I20260811 00:19:37.679943 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:37.661675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.16.71.190:39155)
0811 00:19:37.662854 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:37.662880 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:37.662983 (+ 103us) server_negotiation.cc:407] Connection header received
0811 00:19:37.665505 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:37.665549 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:37.666367 (+ 818us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:37.666934 (+ 567us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:37.668338 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:37.670378 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:37.672409 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:37.673749 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:37.673974 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:37.675963 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:37.675976 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:37.675999 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:19:37.676849 (+ 850us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:37.676983 (+ 134us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:19:37.677020 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:37.677133 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:37.678020 (+ 887us) server_negotiation.cc:299] Negotiation successful
0811 00:19:37.678449 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":397,"threads_started":1}
I20260811 00:19:37.680038 26284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:37.681555 26284 log.cc:824] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:37.684859 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:19:37.685127 26284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6: No bootstrap required, opened a new log
I20260811 00:19:37.693859 26284 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39155 } }
I20260811 00:19:37.694325 26284 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:37.694403 26284 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c5d5dfd9684c719a11b5fe16a86bc6, State: Initialized, Role: FOLLOWER
I20260811 00:19:37.694990 26284 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39155 } }
I20260811 00:19:37.695219 26284 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:37.695364 26284 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:37.695537 26284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:37.698215 26284 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39155 } }
I20260811 00:19:37.698873 26284 leader_election.cc:302] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2c5d5dfd9684c719a11b5fe16a86bc6; no voters:
I20260811 00:19:37.699676 26284 leader_election.cc:288] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:37.699880 26292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:37.700783 26292 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [term 1 LEADER]: Becoming Leader. State: Replica: e2c5d5dfd9684c719a11b5fe16a86bc6, State: Running, Role: LEADER
I20260811 00:19:37.701360 26292 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39155 } }
I20260811 00:19:37.702140 26284 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:37.704447 26293 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39155 } } }
I20260811 00:19:37.704948 26293 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:37.704756 26294 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c5d5dfd9684c719a11b5fe16a86bc6. Latest consensus state: current_term: 1 leader_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c5d5dfd9684c719a11b5fe16a86bc6" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39155 } } }
I20260811 00:19:37.705097 26294 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:37.707513 26302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:37.714066 26302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:37.720940 26318 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:19:37.726213 26302 catalog_manager.cc:1347] Generated new cluster ID: 0f804ced73b642c6b70685dbdd9b97d3
I20260811 00:19:37.726366 26302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:37.749243 26302 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:37.751255 26302 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:37.765443 26302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2c5d5dfd9684c719a11b5fe16a86bc6: Generated new TSK 0
I20260811 00:19:37.766769 26302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:19:38.226795 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:37.793529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.16.71.190:39155)
0811 00:19:37.793758 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:37.793765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:37.794118 (+ 353us) server_negotiation.cc:407] Connection header received
0811 00:19:37.794359 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:37.794365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:37.794478 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:37.794676 (+ 198us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:19:37.795190 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:37.796358 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:37.797993 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:37.798727 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:37.798866 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:37.799471 (+ 605us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:19:38.226087 (+426616us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 00:19:38.226545 (+ 458us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45516: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 00:19:38.226923 26289 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45516: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 00:19:38.227095 26327 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:37.793369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39155 (local address 127.0.0.1:45516)
0811 00:19:37.794003 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:37.794020 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:37.794162 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:37.794722 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:37.794726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:37.794738 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:19:37.795024 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:37.795034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:37.796585 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:37.796590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:37.797818 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:37.797841 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:37.798959 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:37.798964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:37.799099 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:37.799140 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:19:38.226415 (+427275us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 00:19:38.226701 (+ 286us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.71.190:39155: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":325,"threads_started":1}
I20260811 00:19:38.228327 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 26162
I20260811 00:19:38.229712 26270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:38.334491 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 26162
2026-08-11T00:19:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1235 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (451 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (483 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:39 dist-test-slave-f7th krb5kdc[26353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:39 dist-test-slave-f7th krb5kdc[26353](info): set up 2 sockets
Aug 11 00:19:39 dist-test-slave-f7th krb5kdc[26353](info): commencing operation
krb5kdc: starting...
W20260811 00:19:40.570592 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:40 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:19:40.838166 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:32977
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:32977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:41.204414 26371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:41.204803 26371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:41.204905 26371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:41.212519 26371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:41.212711 26371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:41.212779 26371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:41.212838 26371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:41.212886 26371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:41.224017 26371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:32977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:32977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:41.225551 26371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:41.227064 26371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:41.240197 26377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:41.240280 26381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:41.240232 26378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:41.241933 26371 server_base.cc:1034] running on GCE node
I20260811 00:19:41.242897 26371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:41.244488 26371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:41.245762 26371 hybrid_clock.cc:648] HybridClock initialized: now 1786407581245698 us; error 49 us; skew 500 ppm
I20260811 00:19:41.246309 26371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:41 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407581, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:41.250761 26371 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:19:41.254925 26371 webserver.cc:460] Webserver started at http://127.16.71.190:38383/ using document root <none> and password file <none>
I20260811 00:19:41.255926 26371 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:41.256057 26371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:41.256356 26371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:41.259723 26371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "e045698172794020a6e32ee59c13b6d1"
format_stamp: "Formatted at 2026-08-11 00:19:41 on dist-test-slave-f7th"
server_key: "b17f701780f155edf0c401ecdfda4974"
server_key_iv: "29569a821c5f7b0ea3641e5661eae9c6"
server_key_version: "encryptionkey@0"
I20260811 00:19:41.260561 26371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "e045698172794020a6e32ee59c13b6d1"
format_stamp: "Formatted at 2026-08-11 00:19:41 on dist-test-slave-f7th"
server_key: "b17f701780f155edf0c401ecdfda4974"
server_key_iv: "29569a821c5f7b0ea3641e5661eae9c6"
server_key_version: "encryptionkey@0"
I20260811 00:19:41.266278 26371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:19:41.270351 26391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:41.271931 26371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260811 00:19:41.272127 26371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "e045698172794020a6e32ee59c13b6d1"
format_stamp: "Formatted at 2026-08-11 00:19:41 on dist-test-slave-f7th"
server_key: "b17f701780f155edf0c401ecdfda4974"
server_key_iv: "29569a821c5f7b0ea3641e5661eae9c6"
server_key_version: "encryptionkey@0"
I20260811 00:19:41.272328 26371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:41.293641 26371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:41.298499 26371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:41.298771 26371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:41.317274 26371 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:32977
I20260811 00:19:41.317318 26489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:32977 every 8 connection(s)
I20260811 00:19:41.318746 26371 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:41.323002 26490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:41.326009 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 26371
I20260811 00:19:41.326272 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:41.327088 16670 external_mini_cluster.cc:1428] Setting key 9b555a3daadb7fc7daee2bc6f5f0635e
I20260811 00:19:41.337911 26490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1: Bootstrap starting.
I20260811 00:19:41.341773 26490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1: Neither blocks nor log segments found. Creating new log.
Aug 11 00:19:41 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:41.343072 26490 log.cc:824] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:41.347267 26490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1: No bootstrap required, opened a new log
I20260811 00:19:41.354621 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:41.329771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.16.71.190:32977)
0811 00:19:41.330772 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:41.330785 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:41.330833 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:19:41.332749 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:41.332778 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:41.333255 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:41.333693 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:41.335385 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:41.336965 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:41.338461 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:41.339330 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:41.339554 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:41.344701 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:41.344716 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:41.344740 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:41.344792 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:41.348957 (+ 4165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:41.349913 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:41.349929 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:41.349939 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:41.350183 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:41.350730 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:41.350736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:41.350739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:41.351615 (+ 876us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:41.351966 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:41.352500 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 00:19:41.353118 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":461,"threads_started":1}
I20260811 00:19:41.357761 26490 raft_consensus.cc:348] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e045698172794020a6e32ee59c13b6d1" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32977 } }
I20260811 00:19:41.358239 26490 raft_consensus.cc:374] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:41.358355 26490 raft_consensus.cc:729] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e045698172794020a6e32ee59c13b6d1, State: Initialized, Role: FOLLOWER
I20260811 00:19:41.359125 26490 consensus_queue.cc:260] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e045698172794020a6e32ee59c13b6d1" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32977 } }
I20260811 00:19:41.359400 26490 raft_consensus.cc:388] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:41.359556 26490 raft_consensus.cc:482] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:41.359719 26490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:41.362242 26490 raft_consensus.cc:504] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e045698172794020a6e32ee59c13b6d1" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32977 } }
I20260811 00:19:41.362928 26490 leader_election.cc:302] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e045698172794020a6e32ee59c13b6d1; no voters:
I20260811 00:19:41.363723 26490 leader_election.cc:288] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:41.363821 26495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:41.364512 26495 raft_consensus.cc:686] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [term 1 LEADER]: Becoming Leader. State: Replica: e045698172794020a6e32ee59c13b6d1, State: Running, Role: LEADER
I20260811 00:19:41.365073 26495 consensus_queue.cc:237] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e045698172794020a6e32ee59c13b6d1" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32977 } }
I20260811 00:19:41.365911 26490 sys_catalog.cc:564] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:41.367998 26497 sys_catalog.cc:455] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e045698172794020a6e32ee59c13b6d1. Latest consensus state: current_term: 1 leader_uuid: "e045698172794020a6e32ee59c13b6d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e045698172794020a6e32ee59c13b6d1" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32977 } } }
I20260811 00:19:41.368403 26497 sys_catalog.cc:458] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:41.369778 26496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e045698172794020a6e32ee59c13b6d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e045698172794020a6e32ee59c13b6d1" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32977 } } }
I20260811 00:19:41.370157 26496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:41.371313 26502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:41.378183 26502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:41.389889 26502 catalog_manager.cc:1347] Generated new cluster ID: 927942e0310847b2b52cc951cb346829
I20260811 00:19:41.390039 26502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:41.403240 26502 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:41.404757 26502 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:41.419803 26502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e045698172794020a6e32ee59c13b6d1: Generated new TSK 0
I20260811 00:19:41.421041 26502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:19:41.504693 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32977
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:41.927716 26529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:41.928046 26529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:41.928184 26529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:41.935950 26529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:41.936106 26529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:41.936265 26529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:41.946843 26529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:41.948307 26529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:41.949753 26529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:41.963950 26534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:41.963912 26536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:41.963996 26540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:41.964890 26529 server_base.cc:1034] running on GCE node
I20260811 00:19:41.965633 26529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:41.966784 26529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:41.968063 26529 hybrid_clock.cc:648] HybridClock initialized: now 1786407581967982 us; error 72 us; skew 500 ppm
I20260811 00:19:41.968454 26529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:41 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407581, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:41.975502 26529 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:19:41.977407 26529 webserver.cc:460] Webserver started at http://127.16.71.129:41813/ using document root <none> and password file <none>
I20260811 00:19:41.978192 26529 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:41.978340 26529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:41.978679 26529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:41.982028 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "3c172c2e443e439da35b3ee2a0f76d26"
format_stamp: "Formatted at 2026-08-11 00:19:41 on dist-test-slave-f7th"
server_key: "8d9d2149b066794bd6a60e1ae3dc1cbe"
server_key_iv: "afd88028dbe1e9fe907faf93ca7eeab8"
server_key_version: "encryptionkey@0"
I20260811 00:19:41.982906 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "3c172c2e443e439da35b3ee2a0f76d26"
format_stamp: "Formatted at 2026-08-11 00:19:41 on dist-test-slave-f7th"
server_key: "8d9d2149b066794bd6a60e1ae3dc1cbe"
server_key_iv: "afd88028dbe1e9fe907faf93ca7eeab8"
server_key_version: "encryptionkey@0"
I20260811 00:19:41.989229 26529 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:19:41.993383 26551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:41.994853 26529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:19:41.995071 26529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "3c172c2e443e439da35b3ee2a0f76d26"
format_stamp: "Formatted at 2026-08-11 00:19:41 on dist-test-slave-f7th"
server_key: "8d9d2149b066794bd6a60e1ae3dc1cbe"
server_key_iv: "afd88028dbe1e9fe907faf93ca7eeab8"
server_key_version: "encryptionkey@0"
I20260811 00:19:41.995302 26529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:42.012682 26529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:42.018532 26529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:42.018860 26529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:42.020092 26529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:42.021821 26529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:42.021929 26529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:42.022073 26529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:42.022133 26529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:42.056389 26529 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:36851
I20260811 00:19:42.056427 26749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:36851 every 8 connection(s)
I20260811 00:19:42.057796 26529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:42.067287 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 26529
I20260811 00:19:42.067512 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:42.068162 16670 external_mini_cluster.cc:1428] Setting key a7b70b639a4c5361fc8c2430c9f63694
Aug 11 00:19:42 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407581, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:42.080214 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:42.060590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:46621 (local address 127.16.71.190:32977)
0811 00:19:42.061121 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:42.061126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:42.062042 (+ 916us) server_negotiation.cc:407] Connection header received
0811 00:19:42.063998 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:42.064003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:42.064095 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:42.064195 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:42.066030 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.066788 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.068942 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.069369 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.069483 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:42.073946 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:42.073952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:42.073956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:42.073995 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:42.076721 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:42.077669 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:42.077674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:42.077676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:42.077854 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:42.078317 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:42.078321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:42.078323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:42.078623 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:42.078764 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:42.079548 (+ 784us) server_negotiation.cc:299] Negotiation successful
0811 00:19:42.079805 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":277,"threads_started":1}
I20260811 00:19:42.081313 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:42.061064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:32977 (local address 127.16.71.129:46621)
0811 00:19:42.061862 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:42.061911 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:42.063670 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:42.064504 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:42.064514 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:42.065133 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:42.065799 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:42.065845 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.067046 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.067052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:42.068758 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:42.068776 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.069638 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.069647 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:42.069980 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:42.070688 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:42.070723 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:42.073658 (+ 2935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:42.076917 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:42.076926 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:42.076942 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:42.077509 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:42.077977 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:42.077981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:42.077985 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:42.078195 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:42.078756 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:42.078764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:42.079159 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 00:19:42.079416 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 00:19:42.080025 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":327,"threads_started":1}
I20260811 00:19:42.083067 26750 heartbeater.cc:347] Connected to a master server at 127.16.71.190:32977
I20260811 00:19:42.083395 26750 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:42.084179 26750 heartbeater.cc:511] Master 127.16.71.190:32977 requested a full tablet report, sending...
I20260811 00:19:42.086834 26419 ts_manager.cc:194] Registered new tserver with Master: 3c172c2e443e439da35b3ee2a0f76d26 (127.16.71.129:36851)
I20260811 00:19:42.088941 26419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:46621
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:19:42.125787 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32977
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:42.484802 26758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:42.485139 26758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:42.485263 26758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:42.493518 26758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:42.493659 26758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:42.493824 26758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:42.504374 26758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:42.505900 26758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:42.507450 26758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:42.521906 26765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:42.521870 26769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:42.521935 26767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:42.522809 26758 server_base.cc:1034] running on GCE node
I20260811 00:19:42.523561 26758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:42.524714 26758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:42.525966 26758 hybrid_clock.cc:648] HybridClock initialized: now 1786407582525908 us; error 49 us; skew 500 ppm
I20260811 00:19:42.526320 26758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:42 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407582, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:42.530243 26758 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:19:42.532160 26758 webserver.cc:460] Webserver started at http://127.16.71.130:37747/ using document root <none> and password file <none>
I20260811 00:19:42.533056 26758 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:42.533214 26758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:42.533571 26758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:42.537063 26758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "552d10e7d3f74307b630341a3bf6ec83"
format_stamp: "Formatted at 2026-08-11 00:19:42 on dist-test-slave-f7th"
server_key: "b9585524f73f91dec25dc1f339c93a89"
server_key_iv: "368b6d42c29c5624c73b3c9c4cc26d58"
server_key_version: "encryptionkey@0"
I20260811 00:19:42.538030 26758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "552d10e7d3f74307b630341a3bf6ec83"
format_stamp: "Formatted at 2026-08-11 00:19:42 on dist-test-slave-f7th"
server_key: "b9585524f73f91dec25dc1f339c93a89"
server_key_iv: "368b6d42c29c5624c73b3c9c4cc26d58"
server_key_version: "encryptionkey@0"
I20260811 00:19:42.544359 26758 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:19:42.548506 26787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:42.550179 26758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:42.550390 26758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "552d10e7d3f74307b630341a3bf6ec83"
format_stamp: "Formatted at 2026-08-11 00:19:42 on dist-test-slave-f7th"
server_key: "b9585524f73f91dec25dc1f339c93a89"
server_key_iv: "368b6d42c29c5624c73b3c9c4cc26d58"
server_key_version: "encryptionkey@0"
I20260811 00:19:42.550634 26758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:42.569878 26758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:42.575062 26758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:42.575341 26758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:42.576396 26758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:42.578107 26758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:42.578204 26758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:42.578305 26758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:42.578351 26758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:42.610888 26758 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:42565
I20260811 00:19:42.610942 26988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:42565 every 8 connection(s)
I20260811 00:19:42.612412 26758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:42.622700 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 26758
I20260811 00:19:42.622969 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:42.623432 16670 external_mini_cluster.cc:1428] Setting key 93727f0edd15bbf4e877ebd913e310a3
Aug 11 00:19:42 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407582, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:42.638525 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:42.615852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:52319 (local address 127.16.71.190:32977)
0811 00:19:42.616446 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:42.616451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:42.618440 (+ 1989us) server_negotiation.cc:407] Connection header received
0811 00:19:42.620464 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:42.620469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:42.620563 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:42.620716 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:42.622870 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.623799 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.626271 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.626978 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.627122 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:42.631401 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:42.631408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:42.631413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:42.631459 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:42.634799 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:42.635751 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:42.635758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:42.635760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:42.635908 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:42.636458 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:42.636460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:42.636462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:42.636829 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:42.636955 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:42.637875 (+ 920us) server_negotiation.cc:299] Negotiation successful
0811 00:19:42.638147 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":293,"threads_started":1}
I20260811 00:19:42.639765 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:42.616591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:32977 (local address 127.16.71.130:52319)
0811 00:19:42.618221 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:42.618271 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:42.620127 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:42.621019 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:42.621030 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:42.621848 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:42.622585 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:42.622609 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.624032 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.624037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:42.626022 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:42.626050 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:42.627283 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:42.627292 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:42.627605 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:42.628334 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:42.628367 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:42.631031 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:42.635012 (+ 3981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:42.635022 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:42.635039 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:42.635585 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:42.636024 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:42.636029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:42.636032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:42.636322 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:42.636965 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:42.636975 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:42.637459 (+ 484us) client_negotiation.cc:769] Sending connection context
0811 00:19:42.637717 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 00:19:42.638275 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"thread_start_us":995,"threads_started":1}
I20260811 00:19:42.641503 26989 heartbeater.cc:347] Connected to a master server at 127.16.71.190:32977
I20260811 00:19:42.641888 26989 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:42.642933 26989 heartbeater.cc:511] Master 127.16.71.190:32977 requested a full tablet report, sending...
I20260811 00:19:42.644884 26419 ts_manager.cc:194] Registered new tserver with Master: 552d10e7d3f74307b630341a3bf6ec83 (127.16.71.130:42565)
I20260811 00:19:42.645807 26419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:52319
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:19:42.684705 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32977
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:43.053555 26996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:43.053869 26996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:43.053972 26996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:43.062148 26996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:43.062286 26996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:43.062438 26996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:43.073446 26996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:43.074888 26996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:43.076361 26996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:43.091055 27005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:43.091014 27007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:43.091078 27003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:43.091977 26996 server_base.cc:1034] running on GCE node
I20260811 00:19:43.092378 26750 heartbeater.cc:503] Master 127.16.71.190:32977 was elected leader, sending a full tablet report...
I20260811 00:19:43.092787 26996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:43.093948 26996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:43.095178 26996 hybrid_clock.cc:648] HybridClock initialized: now 1786407583095099 us; error 69 us; skew 500 ppm
I20260811 00:19:43.095553 26996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.099161 26996 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:19:43.100917 26996 webserver.cc:460] Webserver started at http://127.16.71.131:33245/ using document root <none> and password file <none>
I20260811 00:19:43.101681 26996 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:43.101820 26996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:43.102145 26996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:43.105454 26996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "be1740781a7d45faacfaf3c019cc9955"
format_stamp: "Formatted at 2026-08-11 00:19:43 on dist-test-slave-f7th"
server_key: "32d3440563fd25388300510e7d201449"
server_key_iv: "e56873e01b00960a00a16b5f88708925"
server_key_version: "encryptionkey@0"
I20260811 00:19:43.106432 26996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "be1740781a7d45faacfaf3c019cc9955"
format_stamp: "Formatted at 2026-08-11 00:19:43 on dist-test-slave-f7th"
server_key: "32d3440563fd25388300510e7d201449"
server_key_iv: "e56873e01b00960a00a16b5f88708925"
server_key_version: "encryptionkey@0"
I20260811 00:19:43.112602 26996 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:19:43.116783 27026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:43.118319 26996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:19:43.118506 26996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "be1740781a7d45faacfaf3c019cc9955"
format_stamp: "Formatted at 2026-08-11 00:19:43 on dist-test-slave-f7th"
server_key: "32d3440563fd25388300510e7d201449"
server_key_iv: "e56873e01b00960a00a16b5f88708925"
server_key_version: "encryptionkey@0"
I20260811 00:19:43.118731 26996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:43.133322 26996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:43.137746 26996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:43.138052 26996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:43.139135 26996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:43.140704 26996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:43.140794 26996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:43.140928 26996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:43.140985 26996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:43.173738 26996 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:38099
I20260811 00:19:43.173799 27224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:38099 every 8 connection(s)
I20260811 00:19:43.175194 26996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:43.183395 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 26996
I20260811 00:19:43.183610 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:43.184041 16670 external_mini_cluster.cc:1428] Setting key 18f96e2f49d70f12a92a7b24570a3e63
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:19:43.197132 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.178059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:38353 (local address 127.16.71.190:32977)
0811 00:19:43.178521 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.178526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.179444 (+ 918us) server_negotiation.cc:407] Connection header received
0811 00:19:43.181435 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.181441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.181551 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.181651 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.183492 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.184351 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.186782 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.187268 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.187425 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.191349 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:43.191355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:43.191358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:43.191398 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:43.194035 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.194871 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.194875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.194877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.195029 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.195426 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.195430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.195432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.195720 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:43.195843 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:43.196572 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 00:19:43.196823 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":236,"threads_started":1}
I20260811 00:19:43.198393 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.178494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:32977 (local address 127.16.71.131:38353)
0811 00:19:43.179269 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:43.179320 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:43.181117 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:43.181951 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:43.181963 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:43.182578 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:43.183277 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:43.183299 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.184549 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.184555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:43.186612 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:43.186631 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.187531 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.187541 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:43.187860 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.188522 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:43.188548 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:43.191031 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:43.194242 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:43.194252 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:43.194267 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:43.194757 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:43.195124 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:43.195127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:43.195129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:43.195338 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:43.195814 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:43.195820 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:43.196206 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 00:19:43.196449 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 00:19:43.196952 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":295,"threads_started":1}
I20260811 00:19:43.200050 27225 heartbeater.cc:347] Connected to a master server at 127.16.71.190:32977
I20260811 00:19:43.200404 27225 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:43.201303 27225 heartbeater.cc:511] Master 127.16.71.190:32977 requested a full tablet report, sending...
I20260811 00:19:43.202960 26417 ts_manager.cc:194] Registered new tserver with Master: be1740781a7d45faacfaf3c019cc9955 (127.16.71.131:38099)
I20260811 00:19:43.203895 26417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:38353
I20260811 00:19:43.213747 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/test-admin.keytab.
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.247419 16670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 00:19:43.254721 27238 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:43.252106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57588 (local address 127.16.71.129:36851)
0811 00:19:43.252829 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.252836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.252866 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:19:43.252932 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.252940 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.253163 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.253302 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.254300 (+ 998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57588: BlockingRecv error: recv got EOF from 127.0.0.1:57588 (error 108)
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":372,"threads_started":1}
W20260811 00:19:43.255259 16670 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc' of type FILE
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.257067 16670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.258603 16670 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:43.275557 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.260536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57602 (local address 127.16.71.129:36851)
0811 00:19:43.260810 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.260817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.261000 (+ 183us) server_negotiation.cc:407] Connection header received
0811 00:19:43.261239 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.261244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.261313 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.261390 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.262419 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.263423 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.264740 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.265342 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.265448 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.269492 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:43.269517 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:43.269536 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:43.269580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:43.272455 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.273415 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.273424 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.273426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.273597 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.274029 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.274032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.274034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.274419 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:43.274691 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:43.275022 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 00:19:43.275246 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260811 00:19:43.282510 27238 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:43.280478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57604 (local address 127.16.71.129:36851)
0811 00:19:43.280743 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.280749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.280953 (+ 204us) server_negotiation.cc:407] Connection header received
0811 00:19:43.281147 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.281152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.281229 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.281312 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.282257 (+ 945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57604: BlockingRecv error: recv got EOF from 127.0.0.1:57604 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 00:19:43.283021 16670 init.cc:244] Error while opening credential cache '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc)
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.284731 16670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.286194 16670 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:43.302243 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.287969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.16.71.129:36851)
0811 00:19:43.288217 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.288223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.288327 (+ 104us) server_negotiation.cc:407] Connection header received
0811 00:19:43.288539 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.288543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.288610 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.288733 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.289812 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.290555 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.291875 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.292319 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.292440 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.296418 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:43.296424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:43.296427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:43.296485 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:43.299296 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.300127 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.300133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.300136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.300288 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.300724 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.300729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.300733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.301104 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:43.301384 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:43.301731 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 00:19:43.301963 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 00:19:43.307039 27238 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:43.305154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57620 (local address 127.16.71.129:36851)
0811 00:19:43.305321 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.305327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.305644 (+ 317us) server_negotiation.cc:407] Connection header received
0811 00:19:43.305792 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.305795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.305869 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.305964 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.306768 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57620: BlockingRecv error: recv got EOF from 127.0.0.1:57620 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 00:19:43.307242 16670 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407527335556-16670-0/krb5kdc/krb5cc' of type FILE
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.308817 16670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:43.310137 16670 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:19:43.326669 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.311647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.16.71.129:36851)
0811 00:19:43.311895 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.311902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.312018 (+ 116us) server_negotiation.cc:407] Connection header received
0811 00:19:43.312189 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.312193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.312258 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.312344 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.313406 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.314139 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.315474 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.315889 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.315988 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.320553 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:43.320559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:43.320563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:43.320608 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:43.323548 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.324500 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.324505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.324507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.324720 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.325183 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.325189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.325193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.325545 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:43.325840 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:43.326126 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 00:19:43.326342 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 00:19:43.328438 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 26529
I20260811 00:19:43.341779 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.329284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.16.71.129:36851)
0811 00:19:43.329587 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.329593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.329811 (+ 218us) server_negotiation.cc:407] Connection header received
0811 00:19:43.329981 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.329987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.330055 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.330140 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.331173 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.331868 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.333125 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.333588 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.333704 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.335932 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:43.335937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:43.335941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:43.335980 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:43.338850 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.339645 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.339650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.339652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.339812 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.340217 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.340221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.340223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.340541 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:43.340846 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:43.341192 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 00:19:43.341451 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:19:43.342464 26743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:43.428121 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 26529
I20260811 00:19:43.454249 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 26758
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:19:43.472250 27258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.455203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.16.71.130:42565)
0811 00:19:43.455820 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.455829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.455860 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:19:43.455951 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.455961 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.456334 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.456502 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.457647 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.458736 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.460178 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.461054 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.461172 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.465456 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:43.465473 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:43.465492 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:43.465562 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:43.468857 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.469738 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.469746 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.469749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.469894 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.470387 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.470392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.470394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.470973 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:43.471254 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:43.471698 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 00:19:43.471936 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":319,"threads_started":1}
I20260811 00:19:43.473723 26979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:43.556870 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 26758
I20260811 00:19:43.582492 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 26996
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[26353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:19:43.599943 27260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:43.583680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.16.71.131:38099)
0811 00:19:43.584386 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:43.584393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:43.584427 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:19:43.584488 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:43.584495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:43.584904 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:43.585047 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:43.586057 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.587150 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.588493 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:43.589264 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:43.589385 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:43.593715 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:43.593733 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:43.593751 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:43.593797 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:43.596856 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.597719 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.597728 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.597730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.597879 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:43.598284 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:43.598289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:43.598291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:43.598728 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:43.599005 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:43.599312 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 00:19:43.599610 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":370,"threads_started":1}
I20260811 00:19:43.601356 27218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:43.686519 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 26996
I20260811 00:19:43.717404 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 26371
I20260811 00:19:43.718688 26481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:43.824172 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 26371
2026-08-11T00:19:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4560 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[27269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[27269](info): set up 2 sockets
Aug 11 00:19:43 dist-test-slave-f7th krb5kdc[27269](info): commencing operation
krb5kdc: starting...
W20260811 00:19:45.114887 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:45 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab.
Entry for principal oryx/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab.
I20260811 00:19:45.385332 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42327
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab
--principal=oryx/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:42327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:45.749249 27290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:45.749585 27290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:45.749707 27290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:45.757505 27290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:45.757658 27290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:45.757827 27290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:45.757965 27290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:45.758044 27290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:45.769230 27290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:42327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab
--principal=oryx/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:45.770808 27290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:45.772311 27290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:45.786145 27297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:45.786163 27299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:45.786212 27303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:45.787175 27290 server_base.cc:1034] running on GCE node
I20260811 00:19:45.788126 27290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:45.789855 27290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:45.791107 27290 hybrid_clock.cc:648] HybridClock initialized: now 1786407585791049 us; error 49 us; skew 500 ppm
I20260811 00:19:45.791527 27290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:45 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:45.795056 27290 init.cc:377] Logged in from keytab as oryx/127.16.71.190@KRBTEST.COM (short username oryx)
I20260811 00:19:45.796970 27290 webserver.cc:460] Webserver started at http://127.16.71.190:35545/ using document root <none> and password file <none>
I20260811 00:19:45.797760 27290 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:45.797899 27290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:45.798252 27290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:45.801656 27290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "2dada80505ec4b0f87382962092f83b7"
format_stamp: "Formatted at 2026-08-11 00:19:45 on dist-test-slave-f7th"
server_key: "697d9a0e033b3bbc32371c409c7cccf5"
server_key_iv: "a5df4f2c985413a99198dd63e1e2c019"
server_key_version: "encryptionkey@0"
I20260811 00:19:45.802551 27290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "2dada80505ec4b0f87382962092f83b7"
format_stamp: "Formatted at 2026-08-11 00:19:45 on dist-test-slave-f7th"
server_key: "697d9a0e033b3bbc32371c409c7cccf5"
server_key_iv: "a5df4f2c985413a99198dd63e1e2c019"
server_key_version: "encryptionkey@0"
I20260811 00:19:45.808542 27290 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:19:45.812556 27320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:45.814265 27290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:45.814478 27290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "2dada80505ec4b0f87382962092f83b7"
format_stamp: "Formatted at 2026-08-11 00:19:45 on dist-test-slave-f7th"
server_key: "697d9a0e033b3bbc32371c409c7cccf5"
server_key_iv: "a5df4f2c985413a99198dd63e1e2c019"
server_key_version: "encryptionkey@0"
I20260811 00:19:45.814703 27290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:45.832073 27290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:45.837155 27290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:45.837447 27290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:45.857250 27290 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:42327
I20260811 00:19:45.857275 27417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:42327 every 8 connection(s)
I20260811 00:19:45.858804 27290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:45.862885 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 27290
I20260811 00:19:45.863157 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:45.863173 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:45.863795 16670 external_mini_cluster.cc:1428] Setting key 4357b02429111196181d366ab656e6df
I20260811 00:19:45.878671 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: Bootstrap starting.
Aug 11 00:19:45 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:45.882649 27418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:45.884191 27418 log.cc:824] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:45.888437 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: No bootstrap required, opened a new log
I20260811 00:19:45.890725 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:45.865969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.16.71.190:42327)
0811 00:19:45.866979 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:45.866997 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:45.867068 (+ 71us) server_negotiation.cc:407] Connection header received
0811 00:19:45.868833 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:45.868875 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:45.869516 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:45.870063 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:45.871680 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:45.873636 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:45.875078 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:45.876288 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:45.876521 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:45.880834 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:45.880847 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:45.880868 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:45.880932 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:45.885128 (+ 4196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:45.886215 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:45.886225 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:45.886234 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:45.886488 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:45.886928 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:45.886933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:45.886936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:45.887744 (+ 808us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:45.888250 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:45.888671 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 00:19:45.889216 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":377,"threads_started":1}
I20260811 00:19:45.897662 27418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } }
I20260811 00:19:45.898087 27418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:45.898168 27418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dada80505ec4b0f87382962092f83b7, State: Initialized, Role: FOLLOWER
I20260811 00:19:45.898749 27418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } }
I20260811 00:19:45.898973 27418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:45.899081 27418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:45.899210 27418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:45.901919 27418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } }
I20260811 00:19:45.902468 27418 leader_election.cc:302] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dada80505ec4b0f87382962092f83b7; no voters:
I20260811 00:19:45.903111 27418 leader_election.cc:288] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:45.903256 27427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:45.903944 27427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 1 LEADER]: Becoming Leader. State: Replica: 2dada80505ec4b0f87382962092f83b7, State: Running, Role: LEADER
I20260811 00:19:45.904526 27427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } }
I20260811 00:19:45.905387 27418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:45.907603 27429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dada80505ec4b0f87382962092f83b7. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } }
I20260811 00:19:45.908174 27429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:45.909586 27428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } }
I20260811 00:19:45.910244 27428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:45.914268 27437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:45.922370 27437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:45.933385 27437 catalog_manager.cc:1347] Generated new cluster ID: 51a418b47455482897f68fdfa0314b89
I20260811 00:19:45.933521 27437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:45.954233 27437 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:45.956669 27437 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:45.968703 27437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: Generated new TSK 0
I20260811 00:19:45.969990 27437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab.
Entry for principal oryx/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab.
I20260811 00:19:46.047147 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42327
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab
--principal=oryx/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:46.409833 27462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:46.410166 27462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:46.410288 27462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:46.418056 27462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:46.418188 27462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:46.418332 27462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:46.429289 27462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab
--principal=oryx/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:46.430724 27462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:46.432202 27462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:46.447245 27475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:46.447172 27473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:46.447216 27480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:46.448102 27462 server_base.cc:1034] running on GCE node
I20260811 00:19:46.448923 27462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:46.450143 27462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:46.451426 27462 hybrid_clock.cc:648] HybridClock initialized: now 1786407586451375 us; error 48 us; skew 500 ppm
I20260811 00:19:46.451793 27462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:46 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407586, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:46.455873 27462 init.cc:377] Logged in from keytab as oryx/127.16.71.129@KRBTEST.COM (short username oryx)
I20260811 00:19:46.458048 27462 webserver.cc:460] Webserver started at http://127.16.71.129:37721/ using document root <none> and password file <none>
I20260811 00:19:46.459146 27462 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:46.459316 27462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:46.459728 27462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:46.463195 27462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "50104b6eb2364c70a066d1547eeb9591"
format_stamp: "Formatted at 2026-08-11 00:19:46 on dist-test-slave-f7th"
server_key: "4569c7411a94b3c49e645db31ce45b5f"
server_key_iv: "853170b1e7816dd6cec9427ab3f53146"
server_key_version: "encryptionkey@0"
I20260811 00:19:46.464064 27462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "50104b6eb2364c70a066d1547eeb9591"
format_stamp: "Formatted at 2026-08-11 00:19:46 on dist-test-slave-f7th"
server_key: "4569c7411a94b3c49e645db31ce45b5f"
server_key_iv: "853170b1e7816dd6cec9427ab3f53146"
server_key_version: "encryptionkey@0"
I20260811 00:19:46.470312 27462 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:19:46.474377 27492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:46.475932 27462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:19:46.476125 27462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "50104b6eb2364c70a066d1547eeb9591"
format_stamp: "Formatted at 2026-08-11 00:19:46 on dist-test-slave-f7th"
server_key: "4569c7411a94b3c49e645db31ce45b5f"
server_key_iv: "853170b1e7816dd6cec9427ab3f53146"
server_key_version: "encryptionkey@0"
I20260811 00:19:46.476337 27462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:46.491458 27462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:46.496299 27462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:46.496695 27462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:46.497781 27462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:46.499398 27462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:46.499482 27462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:46.499626 27462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:46.499688 27462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:46.533608 27462 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:45607
I20260811 00:19:46.533660 27705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:45607 every 8 connection(s)
I20260811 00:19:46.535004 27462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:46.539443 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 27462
I20260811 00:19:46.539696 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:46.540357 16670 external_mini_cluster.cc:1428] Setting key 6f43ed6b30be99eeb44e779936ce7175
Aug 11 00:19:46 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407586, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:46.556684 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:46.537858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:33655 (local address 127.16.71.190:42327)
0811 00:19:46.538350 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:46.538355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:46.539451 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 00:19:46.541647 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:46.541653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:46.541738 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:46.541835 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:46.543712 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:46.544460 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:46.546184 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:46.546593 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:46.546713 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:46.550607 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:46.550613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:46.550616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:46.550655 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:46.553337 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:46.554227 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:46.554231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:46.554233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:46.554370 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:46.554823 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:46.554826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:46.554828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:46.555107 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:46.555245 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:46.556017 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 00:19:46.556269 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":258,"threads_started":1}
I20260811 00:19:46.557720 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:46.538435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:33655)
0811 00:19:46.539257 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:46.539308 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:46.541346 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:46.542139 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:46.542148 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:46.542856 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:46.543518 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:46.543541 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:46.544680 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:46.544684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:46.546008 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:46.546024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:46.546886 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:46.546896 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:46.547135 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:46.547815 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:46.547840 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:46.550310 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:46.553569 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:46.553577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:46.553593 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:46.554082 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:46.554484 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:46.554490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:46.554492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:46.554704 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:46.555242 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:46.555251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:46.555623 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 00:19:46.555884 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 00:19:46.556477 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":338,"threads_started":1}
I20260811 00:19:46.559439 27707 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42327
I20260811 00:19:46.559882 27707 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:46.560914 27707 heartbeater.cc:511] Master 127.16.71.190:42327 requested a full tablet report, sending...
I20260811 00:19:46.563745 27358 ts_manager.cc:194] Registered new tserver with Master: 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607)
I20260811 00:19:46.565941 27358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:33655
WARNING: no policy specified for oryx/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab.
Entry for principal oryx/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab.
I20260811 00:19:46.600435 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42327
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab
--principal=oryx/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:46.963063 27719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:46.963410 27719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:46.963539 27719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:46.971550 27719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:46.971692 27719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:46.971845 27719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:46.982640 27719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab
--principal=oryx/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:46.984077 27719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:46.985592 27719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:47.000524 27731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:47.000578 27734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:47.000684 27732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:47.001983 27719 server_base.cc:1034] running on GCE node
I20260811 00:19:47.002739 27719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:47.003908 27719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:47.005213 27719 hybrid_clock.cc:648] HybridClock initialized: now 1786407587005155 us; error 56 us; skew 500 ppm
I20260811 00:19:47.005643 27719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:47 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407587, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:47.013509 27719 init.cc:377] Logged in from keytab as oryx/127.16.71.130@KRBTEST.COM (short username oryx)
I20260811 00:19:47.015368 27719 webserver.cc:460] Webserver started at http://127.16.71.130:40965/ using document root <none> and password file <none>
I20260811 00:19:47.016188 27719 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:47.016323 27719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:47.016736 27719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:47.020290 27719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "23c41039d6264036a6970bab6d2280f4"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "569d5cb6b133a97f5e31b2e8c574b9b8"
server_key_iv: "10fb75e77455598509d759715ed7728e"
server_key_version: "encryptionkey@0"
I20260811 00:19:47.021222 27719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "23c41039d6264036a6970bab6d2280f4"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "569d5cb6b133a97f5e31b2e8c574b9b8"
server_key_iv: "10fb75e77455598509d759715ed7728e"
server_key_version: "encryptionkey@0"
I20260811 00:19:47.027556 27719 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:19:47.032167 27743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:47.034178 27719 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:47.034390 27719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "23c41039d6264036a6970bab6d2280f4"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "569d5cb6b133a97f5e31b2e8c574b9b8"
server_key_iv: "10fb75e77455598509d759715ed7728e"
server_key_version: "encryptionkey@0"
I20260811 00:19:47.034606 27719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:47.056126 27719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:47.061148 27719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:47.061453 27719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:47.062526 27719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:47.064059 27719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:47.064143 27719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:47.064258 27719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:47.064309 27719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:47.095952 27719 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:41085
I20260811 00:19:47.095989 27941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:41085 every 8 connection(s)
I20260811 00:19:47.097391 27719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:47.098485 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 27719
I20260811 00:19:47.098713 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:47.099303 16670 external_mini_cluster.cc:1428] Setting key 7cb7769c9b198355741b98c2ef5e9392
Aug 11 00:19:47 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407587, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:47.120904 27947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:47.100407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:42529 (local address 127.16.71.190:42327)
0811 00:19:47.100931 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:47.100937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:47.102168 (+ 1231us) server_negotiation.cc:407] Connection header received
0811 00:19:47.104008 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:47.104014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:47.104096 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:47.104206 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:47.106019 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.107074 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.108842 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.109420 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.109568 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:47.113696 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:47.113702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:47.113706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:47.113747 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:47.116716 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:47.118018 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:47.118023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:47.118025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:47.118197 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:47.118810 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:47.118815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:47.118817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:47.119169 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:47.119297 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:47.120311 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0811 00:19:47.120555 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":280,"threads_started":1}
I20260811 00:19:47.122560 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:47.101120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.130:42529)
0811 00:19:47.101983 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:47.102043 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:47.103725 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:47.104448 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:47.104457 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:47.105062 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:47.105746 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:47.105767 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.107268 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.107273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:47.108658 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:47.108678 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.109704 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.109711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:47.109965 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:47.110700 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:47.110733 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:47.113344 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:47.116964 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:47.116976 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:47.117016 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:47.117815 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:47.118342 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:47.118346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:47.118349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:47.118690 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:47.119347 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:47.119359 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:47.119871 (+ 512us) client_negotiation.cc:769] Sending connection context
0811 00:19:47.120198 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 00:19:47.120882 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":346,"threads_started":1}
I20260811 00:19:47.124330 27943 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42327
I20260811 00:19:47.124711 27943 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:47.125552 27943 heartbeater.cc:511] Master 127.16.71.190:42327 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:19:47.127310 27358 ts_manager.cc:194] Registered new tserver with Master: 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085)
I20260811 00:19:47.128217 27358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:42529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab.
Entry for principal oryx/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab.
I20260811 00:19:47.165249 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42327
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab
--principal=oryx/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:47.522671 27952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:47.523006 27952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:47.523137 27952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:47.530705 27952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:47.530846 27952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:47.531008 27952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:47.542174 27952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab
--principal=oryx/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:47.543596 27952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:47.545032 27952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:47.562811 27958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:47.562881 27960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:47.562911 27967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:47.563498 27952 server_base.cc:1034] running on GCE node
I20260811 00:19:47.564204 27952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:47.565310 27952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:47.566534 27952 hybrid_clock.cc:648] HybridClock initialized: now 1786407587566468 us; error 61 us; skew 500 ppm
I20260811 00:19:47.566885 27952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:19:47.569592 27707 heartbeater.cc:503] Master 127.16.71.190:42327 was elected leader, sending a full tablet report...
Aug 11 00:19:47 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407587, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:47.570431 27952 init.cc:377] Logged in from keytab as oryx/127.16.71.131@KRBTEST.COM (short username oryx)
I20260811 00:19:47.572324 27952 webserver.cc:460] Webserver started at http://127.16.71.131:40265/ using document root <none> and password file <none>
I20260811 00:19:47.573119 27952 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:47.573235 27952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:47.573505 27952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:47.576771 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "9d6fd1186d724dd78559db05d7d42c3a"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "44990e12cd855c506076960f972e9abd"
server_key_iv: "7739758f8ef9c5fa33bad881daef3c7e"
server_key_version: "encryptionkey@0"
I20260811 00:19:47.577654 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "9d6fd1186d724dd78559db05d7d42c3a"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "44990e12cd855c506076960f972e9abd"
server_key_iv: "7739758f8ef9c5fa33bad881daef3c7e"
server_key_version: "encryptionkey@0"
I20260811 00:19:47.583564 27952 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:19:47.587677 27978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:47.589357 27952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:19:47.589547 27952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "9d6fd1186d724dd78559db05d7d42c3a"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "44990e12cd855c506076960f972e9abd"
server_key_iv: "7739758f8ef9c5fa33bad881daef3c7e"
server_key_version: "encryptionkey@0"
I20260811 00:19:47.589759 27952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:47.622363 27952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:47.627390 27952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:47.627712 27952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:47.628854 27952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:47.630391 27952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:47.630489 27952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:47.630620 27952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:47.630671 27952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:47.662065 27952 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:33589
I20260811 00:19:47.662171 28175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:33589 every 8 connection(s)
I20260811 00:19:47.663610 27952 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:47.667302 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 27952
I20260811 00:19:47.667516 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:47.667990 16670 external_mini_cluster.cc:1428] Setting key 6eb32438e7af767a4a5cbc25bd04b097
Aug 11 00:19:47 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407587, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:47.684698 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:47.666297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:46835 (local address 127.16.71.190:42327)
0811 00:19:47.666749 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:47.666754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:47.667819 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 00:19:47.669738 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:47.669743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:47.669855 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:47.669962 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:47.671578 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.672370 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.674075 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.674488 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.674605 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:47.678298 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:47.678304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:47.678307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:47.678345 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:47.681345 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:47.682185 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:47.682189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:47.682191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:47.682325 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:47.682785 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:47.682789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:47.682791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:47.683090 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:47.683241 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:47.684025 (+ 784us) server_negotiation.cc:299] Negotiation successful
0811 00:19:47.684265 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":220,"threads_started":1}
I20260811 00:19:47.685678 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:47.666792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.131:46835)
0811 00:19:47.667608 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:47.667665 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:47.669455 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:47.670229 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:47.670238 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:47.670784 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:47.671397 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:47.671416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.672545 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.672551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:47.673916 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:47.673931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:47.674732 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:47.674740 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:47.674987 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:47.675651 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:47.675680 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:47.678043 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:47.681526 (+ 3483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:47.681536 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:47.681552 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:47.682042 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:47.682445 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:47.682450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:47.682452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:47.682661 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:47.683231 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:47.683240 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:47.683618 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 00:19:47.683874 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 00:19:47.684463 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":330,"threads_started":1}
I20260811 00:19:47.687099 28179 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42327
I20260811 00:19:47.687403 28179 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:47.688184 28179 heartbeater.cc:511] Master 127.16.71.190:42327 requested a full tablet report, sending...
I20260811 00:19:47.689754 27358 ts_manager.cc:194] Registered new tserver with Master: 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
I20260811 00:19:47.690603 27358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:46835
I20260811 00:19:47.695719 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.71.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.189.keytab.
Entry for principal oryx/127.16.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.189.keytab.
I20260811 00:19:47.756552 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:38833
--webserver_interface=127.16.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.189.keytab
--principal=oryx/127.16.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:42327,127.16.71.189:38833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:48.120965 28189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:48.121286 28189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:48.121416 28189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:48.129217 28189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:48.129359 28189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:48.129415 28189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:48.129506 28189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:48.129566 28189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:48.131335 27943 heartbeater.cc:503] Master 127.16.71.190:42327 was elected leader, sending a full tablet report...
I20260811 00:19:48.140540 28189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:42327,127.16.71.189:38833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.189.keytab
--principal=oryx/127.16.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:38833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
W20260811 00:19:48.141000 28189 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.16.71.190:42327,127.16.71.189:38833'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 00:19:48.142115 28189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:48.143491 28189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:48.155751 28200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:48.155781 28203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:48.155987 28199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:48.157032 28189 server_base.cc:1034] running on GCE node
I20260811 00:19:48.158066 28189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:48.159652 28189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:48.160915 28189 hybrid_clock.cc:648] HybridClock initialized: now 1786407588160837 us; error 49 us; skew 500 ppm
I20260811 00:19:48.161265 28189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:48 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:48.164729 28189 init.cc:377] Logged in from keytab as oryx/127.16.71.189@KRBTEST.COM (short username oryx)
I20260811 00:19:48.166458 28189 webserver.cc:460] Webserver started at http://127.16.71.189:44133/ using document root <none> and password file <none>
I20260811 00:19:48.167230 28189 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:48.167344 28189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:48.167639 28189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:48.170929 28189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/instance:
uuid: "fbc8732d8d19493c862e2c1519fae618"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "bb3151aab497ae7a0c085fb2a1e75d59"
server_key_iv: "794e74ee0cc7f31934e113df02fb1228"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.171751 28189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal/instance:
uuid: "fbc8732d8d19493c862e2c1519fae618"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "bb3151aab497ae7a0c085fb2a1e75d59"
server_key_iv: "794e74ee0cc7f31934e113df02fb1228"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.177735 28189 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:19:48.181708 28214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:48.183477 28189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:19:48.183694 28189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
uuid: "fbc8732d8d19493c862e2c1519fae618"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "bb3151aab497ae7a0c085fb2a1e75d59"
server_key_iv: "794e74ee0cc7f31934e113df02fb1228"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.183944 28189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:48.205385 28189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:48.210449 28189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:48.210722 28189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:48.230309 28189 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.189:38833
I20260811 00:19:48.230342 28312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.189:38833 every 8 connection(s)
I20260811 00:19:48.231841 28189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
I20260811 00:19:48.233500 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 28189
I20260811 00:19:48.236716 28314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:48.246724 28314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:19:48 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.189@KRBTEST.COM
Aug 11 00:19:48 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.189@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:48.261432 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.235827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.16.71.189:38833)
0811 00:19:48.236715 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.236731 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.236786 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:19:48.239242 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.239273 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.239863 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.240368 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:48.241929 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.244095 (+ 2166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.245718 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.246913 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.247171 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.252067 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:48.252083 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:48.252100 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:48.252161 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:48.256025 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.257132 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.257142 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.257153 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.257411 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.258001 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.258007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.258011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.258742 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:48.259094 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.259646 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.260271 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":369,"threads_started":1}
I20260811 00:19:48.266799 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.249901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.16.71.190:42327)
0811 00:19:48.250459 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.250465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.251093 (+ 628us) server_negotiation.cc:407] Connection header received
0811 00:19:48.251301 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.251307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.251409 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.251553 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:48.253300 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.254081 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.256231 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.256993 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.257122 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.260784 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:48.260790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:48.260793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:48.260836 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:48.263514 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.264531 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.264536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.264538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.264727 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.265119 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.265124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.265128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.265486 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:48.265637 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.266142 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.266415 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":283,"threads_started":1}
I20260811 00:19:48.266876 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.249877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:35056)
0811 00:19:48.250906 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:48.250974 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:48.251158 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:48.251669 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:48.251675 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:48.252756 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:48.253120 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.253135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.254239 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.254245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.256028 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.256045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.257218 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.257228 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.257430 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.258163 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:48.258182 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:48.260540 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:48.263670 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.263683 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.263691 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.264383 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.264843 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.264849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.264853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.265004 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.265628 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:48.265636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:48.265829 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 00:19:48.266013 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 00:19:48.266420 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":566,"threads_started":1}
I20260811 00:19:48.268853 28314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:19:48 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.189@KRBTEST.COM for oryx/127.16.71.189@KRBTEST.COM
I20260811 00:19:48.283147 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.269705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:42374)
0811 00:19:48.269891 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:48.269907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:48.269997 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:48.270339 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:48.270342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:48.270731 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:48.270881 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.270890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.271956 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.271960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.273020 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.273036 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.273798 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.273803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.273952 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.274579 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:48.274599 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:48.277021 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:48.280395 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.280399 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.280402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.280970 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.281371 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.281374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.281376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.281512 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.282075 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:48.282080 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:48.282231 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 00:19:48.282362 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 00:19:48.282851 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 00:19:48.283491 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.269910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42374 (local address 127.16.71.189:38833)
0811 00:19:48.270104 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.270109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.270126 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:19:48.270169 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.270173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.270231 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.270313 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:48.271027 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.271760 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.273189 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.273631 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.273744 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.277252 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:48.277258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:48.277262 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:48.277298 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:48.280228 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.281098 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.281103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.281105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.281259 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.281637 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.281640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.281642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.281932 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:48.282058 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.282530 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.283039 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":219,"server-negotiator.queue_time_us":51}
I20260811 00:19:48.289255 28314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Bootstrap starting.
I20260811 00:19:48.292796 28314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:48.294018 28314 log.cc:824] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:48.297433 28314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: No bootstrap required, opened a new log
I20260811 00:19:48.304905 28314 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } }
I20260811 00:19:48.305290 28314 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:48.305368 28314 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbc8732d8d19493c862e2c1519fae618, State: Initialized, Role: FOLLOWER
I20260811 00:19:48.306018 28314 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } }
I20260811 00:19:48.307062 28328 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } } }
I20260811 00:19:48.307399 28328 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:19:48.308190 28314 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:19:48.318531 28346 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:19:48.318773 28346 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:19:48.321614 28189 master_runner.cc:428] Detected that this master fbc8732d8d19493c862e2c1519fae618 is joining an existing cluster
I20260811 00:19:48.321740 28189 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.71.189:38833
I20260811 00:19:48.347391 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.329408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.16.71.189:38833)
0811 00:19:48.329657 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.329663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.330790 (+ 1127us) server_negotiation.cc:407] Connection header received
0811 00:19:48.331008 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.331013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.331113 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.331224 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:48.332818 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.333852 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.335746 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.336497 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.336694 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.339627 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:48.339634 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:48.339639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:48.339682 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:48.343825 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.344867 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.344872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.344875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.345053 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.345568 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.345573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.345576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.345983 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:48.346150 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.346687 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.346980 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":67}
I20260811 00:19:48.348402 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.328802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.16.71.190:42327)
0811 00:19:48.328989 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.328995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.330320 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 00:19:48.331253 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.331259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.331364 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.331440 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:48.333155 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.334299 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.337598 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.338237 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.338363 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.341316 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:48.341322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:48.341326 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:48.341368 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:48.344721 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.345862 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.345869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.345873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.346062 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.346554 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.346559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.346563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.346990 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:48.347149 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.347781 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.348065 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 00:19:48.348997 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.329521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:42386)
0811 00:19:48.330658 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:48.330678 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:48.330869 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:48.331391 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:48.331395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:48.332194 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:48.332648 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.332664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.334079 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.334084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.335594 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.335611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.336751 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.336760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.337142 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.337971 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:48.338001 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:48.339401 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:48.343996 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.344001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.344005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.344578 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.345202 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.345206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.345210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.345441 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.346220 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:48.346224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:48.346408 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 00:19:48.347897 (+ 1489us) client_negotiation.cc:240] Negotiation successful
0811 00:19:48.348221 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":803,"threads_started":1}
I20260811 00:19:48.349550 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.328562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:35062)
0811 00:19:48.330169 (+ 1607us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:48.330193 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:48.331071 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:48.331675 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:48.331680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:48.332511 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:48.332946 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.332962 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.334538 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.334543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.337396 (+ 2853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.337410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.338635 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.338642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.338820 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.339638 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:48.339663 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:48.341083 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:48.345179 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.345183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.345187 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.345688 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.346224 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.346228 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.346231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.346414 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.347239 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:48.347244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:48.347466 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 00:19:48.348681 (+ 1215us) client_negotiation.cc:240] Negotiation successful
0811 00:19:48.349042 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1443,"mutex_wait_us":78,"spinlock_wait_cycles":32640,"thread_start_us":1181,"threads_started":1}
WARNING: no policy specified for oryx/127.16.71.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.188@KRBTEST.COM" created.
Aug 11 00:19:48 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.190@KRBTEST.COM for oryx/127.16.71.189@KRBTEST.COM
I20260811 00:19:48.374275 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.357089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.16.71.189:38833)
0811 00:19:48.357297 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.357303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.358462 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 00:19:48.358817 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.358822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.358943 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.359055 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:48.360753 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.361756 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.363814 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.364442 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.364585 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.368314 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:48.368319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:48.368323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:48.368367 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:48.371180 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.371985 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.371990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.371992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.372139 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.372581 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.372585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.372587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.372916 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:48.373058 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.373620 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.373893 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:19:48.374323 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.356699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:42390)
0811 00:19:48.358290 (+ 1591us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:48.358335 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:48.358563 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:48.359195 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:48.359204 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:48.360203 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:48.360528 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.360545 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.361926 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.361933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.363588 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.363608 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.364694 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.364703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.364939 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.365766 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:48.365785 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:48.368086 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:48.371359 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.371378 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.371387 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.371838 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.372258 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.372263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.372267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.372449 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.373062 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:48.373071 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:48.373261 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 00:19:48.373480 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 00:19:48.373888 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"thread_start_us":713,"threads_started":1}
I20260811 00:19:48.376855 27358 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.71.189:38833: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fbc8732d8d19493c862e2c1519fae618"
member_type: NON_VOTER
last_known_addr {
host: "127.16.71.189"
port: 38833
}
attrs {
promote: true
}
}
dest_uuid: "2dada80505ec4b0f87382962092f83b7"
cas_config_opid_index: -1
I20260811 00:19:48.378219 27358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } }
I20260811 00:19:48.386623 28365 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fbc8732d8d19493c862e2c1519fae618 (127.16.71.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } }
I20260811 00:19:48.387975 28269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:48.391402 28269 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Refusing update from remote peer 2dada80505ec4b0f87382962092f83b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 00:19:48.392216 28365 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.71.189:38833
I20260811 00:19:48.392917 28365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:48.393508 28189 master.cc:537] Master@127.16.71.189:38833 shutting down...
I20260811 00:19:48.393675 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } }
I20260811 00:19:48.394380 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
W20260811 00:19:48.401031 27329 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 00:19:48.403694 27329 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 -> Peer fbc8732d8d19493c862e2c1519fae618 (127.16.71.189:38833): Couldn't send request to peer fbc8732d8d19493c862e2c1519fae618. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.188.keytab.
Entry for principal oryx/127.16.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.188.keytab.
I20260811 00:19:48.406193 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:33237
--webserver_interface=127.16.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.188.keytab
--principal=oryx/127.16.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:19:48.416705 28189 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:48.417342 28189 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:48.417559 28189 tablet_replica.cc:330] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: stopping tablet replica
I20260811 00:19:48.424556 28189 master.cc:552] Master@127.16.71.189:38833 shutdown complete.
I20260811 00:19:48.424767 28189 master_runner.cc:305] Clearing existing system tablet
I20260811 00:19:48.425267 28189 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:48.425370 28189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:48.427906 28189 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 00:19:48.429504 28380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:48.429881 28189 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:48.430181 28189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 00:19:48.430274 28189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
uuid: "fbc8732d8d19493c862e2c1519fae618"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "bb3151aab497ae7a0c085fb2a1e75d59"
server_key_iv: "794e74ee0cc7f31934e113df02fb1228"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.434214 28189 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:48.440344 28189 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 00:19:48.440582 28189 log.cc:1196] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 00:19:48.441069 28189 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Deleting consensus metadata
I20260811 00:19:48.441335 28189 master_runner.cc:315] Copying system tablet from 127.16.71.190:42327
I20260811 00:19:48.447388 28189 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: tablet copy: Beginning tablet copy session from remote peer at address 127.16.71.190:42327
I20260811 00:19:48.467134 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.448415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35076 (local address 127.16.71.190:42327)
0811 00:19:48.448640 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.448646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.452900 (+ 4254us) server_negotiation.cc:407] Connection header received
0811 00:19:48.453125 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.453131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.453419 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.453558 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:48.454629 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.455405 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.456802 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.457280 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.457409 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.459680 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:48.459690 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:48.459694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:48.459745 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:48.462779 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.463666 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.463672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.463676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.463894 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:48.465090 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:48.465095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:48.465098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:48.465530 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:48.465686 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.466396 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.466691 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:19:48.468322 28389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.448386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:35076)
0811 00:19:48.452750 (+ 4364us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:48.452771 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:48.452976 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:48.453643 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:48.453648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:48.454190 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:48.454443 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.454455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.455590 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.455595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.456560 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.456577 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.457463 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.457469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.457654 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.458330 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:48.458351 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:48.459353 (+ 1002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:48.462954 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.462964 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.462970 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.463497 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.464085 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:48.464089 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:48.464092 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:48.464918 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:48.465805 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:48.465810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:48.466053 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 00:19:48.467643 (+ 1590us) client_negotiation.cc:240] Negotiation successful
0811 00:19:48.467973 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4162,"thread_start_us":4010,"threads_started":1}
I20260811 00:19:48.469329 27396 tablet_copy_service.cc:140] P 2dada80505ec4b0f87382962092f83b7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fbc8732d8d19493c862e2c1519fae618 ({username='oryx', principal='oryx/127.16.71.189@KRBTEST.COM'} at 127.0.0.1:35076)
I20260811 00:19:48.469740 27396 tablet_copy_service.cc:161] P 2dada80505ec4b0f87382962092f83b7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fbc8732d8d19493c862e2c1519fae618 at {username='oryx', principal='oryx/127.16.71.189@KRBTEST.COM'} at 127.0.0.1:35076: session id = fbc8732d8d19493c862e2c1519fae618-00000000000000000000000000000000
I20260811 00:19:48.473564 27396 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 00:19:48.476300 28189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 00:19:48.481977 28189 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: tablet copy: Starting download of 0 data blocks...
I20260811 00:19:48.482354 28189 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: tablet copy: Starting download of 1 WAL segments...
I20260811 00:19:48.485970 28189 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 00:19:48.489794 27396 tablet_copy_service.cc:342] P 2dada80505ec4b0f87382962092f83b7: Request end of tablet copy session fbc8732d8d19493c862e2c1519fae618-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.71.189@KRBTEST.COM'} at 127.0.0.1:35076
I20260811 00:19:48.490036 27396 tablet_copy_service.cc:433] P 2dada80505ec4b0f87382962092f83b7: ending tablet copy session fbc8732d8d19493c862e2c1519fae618-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fbc8732d8d19493c862e2c1519fae618
I20260811 00:19:48.501966 28189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:48.504791 28390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:48.510672 28391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:48.516124 28189 server_base.cc:1034] running on GCE node
W20260811 00:19:48.517189 28396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:48.517693 28189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:48.518491 28189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:48.519786 28189 hybrid_clock.cc:648] HybridClock initialized: now 1786407588519717 us; error 32 us; skew 500 ppm
I20260811 00:19:48.520046 28189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:48 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:48.522101 28189 init.cc:377] Logged in from keytab as oryx/127.16.71.189@KRBTEST.COM (short username oryx)
I20260811 00:19:48.525632 28189 webserver.cc:460] Webserver started at http://127.16.71.189:45503/ using document root <none> and password file <none>
I20260811 00:19:48.525990 28189 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:48.526087 28189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:48.528647 28189 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 00:19:48.530392 28409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:48.530803 28189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 00:19:48.530951 28189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
uuid: "fbc8732d8d19493c862e2c1519fae618"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "bb3151aab497ae7a0c085fb2a1e75d59"
server_key_iv: "794e74ee0cc7f31934e113df02fb1228"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.531102 28189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:48.554718 28189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:48.558210 28189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:48.585367 28189 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.189:38833
I20260811 00:19:48.585386 28501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.189:38833 every 8 connection(s)
I20260811 00:19:48.586563 28189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
I20260811 00:19:48.592870 28502 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:19:48.598253 28502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Bootstrap starting.
I20260811 00:19:48.614634 28502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:19:48.615299 28502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: Bootstrap complete.
I20260811 00:19:48.616338 28502 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } }
I20260811 00:19:48.616602 28502 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fbc8732d8d19493c862e2c1519fae618, State: Initialized, Role: LEARNER
I20260811 00:19:48.616920 28502 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } }
I20260811 00:19:48.618166 28502 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:48.617875 28509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } }
I20260811 00:19:48.618522 28509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: LEARNER
I20260811 00:19:48.626298 28535 catalog_manager.cc:1259] Loaded cluster ID: 51a418b47455482897f68fdfa0314b89
I20260811 00:19:48.626425 28535 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: loading cluster ID for follower catalog manager: success
I20260811 00:19:48.629154 28535 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: acquiring CA information for follower catalog manager: success
I20260811 00:19:48.631373 28535 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:19:48.693408 28179 heartbeater.cc:503] Master 127.16.71.190:42327 was elected leader, sending a full tablet report...
I20260811 00:19:48.869369 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.860063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:42404)
0811 00:19:48.860379 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:48.860402 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:48.860559 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:48.861649 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:48.861655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:48.861693 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:48.861919 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.861930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.864070 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.864077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.866366 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:48.866388 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.868413 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.868420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:48.868722 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.868809 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 00:19:48.868916 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 00:19:48.869075 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260811 00:19:48.869494 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:48.860390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.16.71.189:38833)
0811 00:19:48.861165 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:48.861170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:48.861200 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:19:48.861280 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:48.861286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:48.861388 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:48.861544 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:48.862182 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.863858 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.866603 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:48.868141 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:48.868332 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:48.868674 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:48.868968 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 00:19:48.869138 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":442,"threads_started":1}
I20260811 00:19:48.871248 28451 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 00:19:48.871976 28509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dada80505ec4b0f87382962092f83b7. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } }
I20260811 00:19:48.872253 28509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: LEARNER
W20260811 00:19:48.884790 28374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:48.885149 28374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:48.885250 28374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:48.893299 28374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:48.893445 28374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:48.893496 28374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:48.893574 28374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:48.893643 28374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:48.905095 28374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.188.keytab
--principal=oryx/127.16.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:33237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:48.906632 28374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:48.908047 28374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:48.921252 28542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:48.921335 28547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:48.921339 28544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:48.923063 28374 server_base.cc:1034] running on GCE node
I20260811 00:19:48.923972 28374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:48.925561 28374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:48.926851 28374 hybrid_clock.cc:648] HybridClock initialized: now 1786407588926761 us; error 62 us; skew 500 ppm
I20260811 00:19:48.927254 28374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:48 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:48.930838 28374 init.cc:377] Logged in from keytab as oryx/127.16.71.188@KRBTEST.COM (short username oryx)
I20260811 00:19:48.932776 28374 webserver.cc:460] Webserver started at http://127.16.71.188:41543/ using document root <none> and password file <none>
I20260811 00:19:48.933549 28374 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:48.933671 28374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:48.933965 28374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:48.937935 28374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/instance:
uuid: "0c67b5d559af48b7bcf14cdfc1f58965"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "040ff36fcbdabc4c1bedc5661c465eb4"
server_key_iv: "42323285e59e26e391277028ada617b6"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.939091 28374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal/instance:
uuid: "0c67b5d559af48b7bcf14cdfc1f58965"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "040ff36fcbdabc4c1bedc5661c465eb4"
server_key_iv: "42323285e59e26e391277028ada617b6"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.945731 28374 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:19:48.950166 28556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:48.952140 28374 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 00:19:48.952356 28374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
uuid: "0c67b5d559af48b7bcf14cdfc1f58965"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "040ff36fcbdabc4c1bedc5661c465eb4"
server_key_iv: "42323285e59e26e391277028ada617b6"
server_key_version: "encryptionkey@0"
I20260811 00:19:48.952596 28374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:48.981784 28374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:48.987634 28374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:48.987950 28374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:49.012485 28374 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.188:33237
I20260811 00:19:49.012543 28658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.188:33237 every 8 connection(s)
I20260811 00:19:49.014156 28374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
I20260811 00:19:49.017506 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 28374
I20260811 00:19:49.018944 28660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:49.029744 28660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.188@KRBTEST.COM
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.188@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:49.044245 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.020216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.16.71.188:33237)
0811 00:19:49.021282 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.021296 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.021342 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:19:49.023047 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.023076 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.023576 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.023956 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.026145 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.027754 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.029563 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.030404 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.030564 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.035763 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.035772 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.035783 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.035833 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.039167 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.039958 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.039968 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.039978 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.040205 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.040654 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.040659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.040663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.041423 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.041674 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.042265 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.042735 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":396,"threads_started":1}
I20260811 00:19:49.049561 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.033072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:35086)
0811 00:19:49.034213 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.034249 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.034425 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.034803 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.034810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.035465 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.035846 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.035858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.037002 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.037006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.038366 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.038381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.039250 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.039256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.039410 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.040054 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.040076 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.042870 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.046772 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.046783 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.046789 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.047284 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.047732 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.047736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.047739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.047888 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.048563 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.048570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.048786 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.048951 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.049254 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":716,"threads_started":1}
I20260811 00:19:49.049609 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.033109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35086 (local address 127.16.71.190:42327)
0811 00:19:49.033813 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.033819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.034342 (+ 523us) server_negotiation.cc:407] Connection header received
0811 00:19:49.034551 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.034557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.034624 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.034712 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.036020 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.036820 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.038514 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.038984 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.039124 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.043101 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.043109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.043114 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.043163 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.046591 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.047453 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.047459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.047463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.047622 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.048028 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.048034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.048037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.048375 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.048575 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.049088 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.049287 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":420,"threads_started":1}
I20260811 00:19:49.051182 28660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.188@KRBTEST.COM for oryx/127.16.71.189@KRBTEST.COM
I20260811 00:19:49.065879 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.051999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:42408)
0811 00:19:49.052254 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.052270 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.052396 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.052800 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.052806 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.053221 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.053379 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.053388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.054437 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.054442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.055407 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.055420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.056125 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.056129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.056261 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.057031 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.057071 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.059772 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.063231 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.063237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.063240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.063724 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.064174 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.064177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.064180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.064320 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.064964 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.064968 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.065128 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.065262 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.065560 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 00:19:49.065917 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.052119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.16.71.189:38833)
0811 00:19:49.052308 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.052313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.052332 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:19:49.052500 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.052504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.052578 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.052710 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.053543 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.054280 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.055551 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.055969 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.056073 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.060044 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.060051 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.060055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.060094 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.063039 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.063896 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.063901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.063903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.064070 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.064433 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.064437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.064439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.064793 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.064927 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.065445 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.065656 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:19:49.067215 28660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407588, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.188@KRBTEST.COM for oryx/127.16.71.188@KRBTEST.COM
I20260811 00:19:49.080466 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.067951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.0.0.1:48212)
0811 00:19:49.068154 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.068178 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.068291 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.068695 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.068698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.069068 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.069229 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.069238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.070367 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.070372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.071264 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.071278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.072084 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.072089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.072228 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.072888 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.072907 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.075080 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.078170 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.078174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.078177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.078591 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.078939 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.078942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.078945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.079103 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.079594 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.079598 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.079759 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.079875 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.080091 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 00:19:49.080677 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.068255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.16.71.188:33237)
0811 00:19:49.068407 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.068412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.068455 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:19:49.068499 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.068502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.068557 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.068671 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.069352 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.070191 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.071415 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.071894 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.072006 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.075304 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.075309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.075312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.075345 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.077994 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.078701 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.078705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.078707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.078837 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.079196 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.079199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.079201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.079482 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.079596 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.079964 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.080238 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":57}
I20260811 00:19:49.085686 28660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Bootstrap starting.
I20260811 00:19:49.089115 28660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:49.090198 28660 log.cc:824] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:49.093279 28660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: No bootstrap required, opened a new log
I20260811 00:19:49.101354 28660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } }
I20260811 00:19:49.101747 28660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:49.101869 28660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c67b5d559af48b7bcf14cdfc1f58965, State: Initialized, Role: FOLLOWER
I20260811 00:19:49.102525 28660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } }
I20260811 00:19:49.103502 28671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } } }
I20260811 00:19:49.103856 28671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:19:49.104573 28660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:19:49.115000 28687 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:19:49.115178 28687 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:19:49.120107 28374 master_runner.cc:428] Detected that this master 0c67b5d559af48b7bcf14cdfc1f58965 is joining an existing cluster
I20260811 00:19:49.120273 28374 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.71.188:33237
I20260811 00:19:49.145777 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.124906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.16.71.189:38833)
0811 00:19:49.125121 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.125128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.125732 (+ 604us) server_negotiation.cc:407] Connection header received
0811 00:19:49.126057 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.126065 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.126158 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.126230 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.128034 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.129299 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.132312 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.132998 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.133110 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.138599 (+ 5489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.138608 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.138613 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.138660 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.141983 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.143010 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.143017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.143022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.143185 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.143798 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.143804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.143807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.144202 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.144333 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.145137 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.145410 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:19:49.145828 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.124701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.16.71.190:42327)
0811 00:19:49.125124 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.125131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.126650 (+ 1519us) server_negotiation.cc:407] Connection header received
0811 00:19:49.126977 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.126983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.127105 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.127192 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.128848 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.130134 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.132688 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.133124 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.133238 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.136656 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.136664 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.136669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.136716 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.141037 (+ 4321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.141963 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.141969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.141972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.142154 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.143793 (+ 1639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.143799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.143802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.144226 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.144377 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.145164 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.145433 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260811 00:19:49.147183 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.125744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:35096)
0811 00:19:49.126526 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.126545 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.126766 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.127441 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.127449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.128213 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.128607 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.128657 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.130443 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.130449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.132505 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.132515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.133336 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.133343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.133565 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.134985 (+ 1420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.135009 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.136380 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.141218 (+ 4838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.141224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.141232 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.141795 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.142318 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.142323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.142327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.143615 (+ 1288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.144505 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.144510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.144842 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.146108 (+ 1266us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.146603 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":408,"spinlock_wait_cycles":42752,"thread_start_us":406,"threads_started":1}
I20260811 00:19:49.147358 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.124839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:42416)
0811 00:19:49.125620 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.125641 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.125865 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.126474 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.126479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.127349 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.127812 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.127828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.129558 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.129564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.131571 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.131589 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.134467 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.134472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.134610 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.136820 (+ 2210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.136842 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.138317 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.142245 (+ 3928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.142251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.142254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.142730 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.143317 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.143321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.143323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.143638 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.144434 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.144438 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.144774 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.146109 (+ 1335us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.146882 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":335,"spinlock_wait_cycles":1920,"thread_start_us":433,"threads_started":1}
I20260811 00:19:49.150758 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 27462
I20260811 00:19:49.151501 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.126725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.16.71.188:33237)
0811 00:19:49.126938 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.126944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.131927 (+ 4983us) server_negotiation.cc:407] Connection header received
0811 00:19:49.132123 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.132129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.132207 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.132313 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.135657 (+ 3344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.136804 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.138811 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.139572 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.139668 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.142297 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.142306 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.142310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.142344 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.146601 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.148929 (+ 2328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.148935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.148938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.149124 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.149530 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.149535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.149538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.149972 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.150165 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.150717 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.151093 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":60}
I20260811 00:19:49.151543 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.127076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.0.0.1:48218)
0811 00:19:49.131799 (+ 4723us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.131819 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.131970 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.133355 (+ 1385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.133360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.134233 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.135480 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.135497 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.137006 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.137011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.138661 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.138675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.140066 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.140074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.140253 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.140825 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.140847 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.141983 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.146783 (+ 4800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.146788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.146792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.148774 (+ 1982us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.149264 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.149268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.149271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.149414 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.150236 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.150241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.150443 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.150591 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.150929 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4553,"mutex_wait_us":1,"thread_start_us":4392,"threads_started":1}
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.190@KRBTEST.COM for oryx/127.16.71.188@KRBTEST.COM
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.129@KRBTEST.COM
I20260811 00:19:49.172820 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.156187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.0.0.1:48230)
0811 00:19:49.156368 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.156387 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.156510 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.157049 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.157054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.157516 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.157748 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.157763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.159218 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.159224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.160709 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.160724 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.161832 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.161838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.162021 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.162610 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.162638 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.165647 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.170015 (+ 4368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.170020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.170023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.170520 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.170905 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.170909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.170911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.171038 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.171661 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.171667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.171865 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.172042 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.172367 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 00:19:49.173381 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.152239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55866 (local address 127.16.71.129:45607)
0811 00:19:49.153066 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.153076 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.153108 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:19:49.153184 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.153193 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.153494 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.155535 (+ 2041us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.155842 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.157502 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.160695 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.161679 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.162476 (+ 797us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.166749 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.166764 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.166781 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.166831 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.169722 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.170610 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.170621 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.170624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.170807 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.171332 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.171337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.171340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.171801 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.172094 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.172447 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.172794 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":425,"threads_started":1}
I20260811 00:19:49.173935 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.156504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48230 (local address 127.16.71.188:33237)
0811 00:19:49.156717 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.156722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.156737 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:19:49.156812 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.156816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.156902 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.156991 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.157971 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.159054 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.161161 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.161663 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.161781 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.165939 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.165947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.165952 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.166014 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.169856 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.170625 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.170631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.170635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.170787 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.171144 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.171149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.171153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.171516 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.171681 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.173315 (+ 1634us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.173569 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:19:49.175335 27690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:49.175438 27358 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.71.188:33237: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965"
member_type: NON_VOTER
last_known_addr {
host: "127.16.71.188"
port: 33237
}
attrs {
promote: true
}
}
dest_uuid: "2dada80505ec4b0f87382962092f83b7"
cas_config_opid_index: 5
I20260811 00:19:49.176761 27358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } }
I20260811 00:19:49.179028 28365 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0c67b5d559af48b7bcf14cdfc1f58965 (127.16.71.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.183131 28365 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.71.188:33237
I20260811 00:19:49.184267 28374 master.cc:537] Master@127.16.71.188:33237 shutting down...
I20260811 00:19:49.185117 28451 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 LEARNER]: Refusing update from remote peer 2dada80505ec4b0f87382962092f83b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 00:19:49.186739 28703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 00:19:49.185111 28702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.191413 28702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:49.200687 28451 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0c67b5d559af48b7bcf14cdfc1f58965 (127.16.71.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.206578 28509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: NON_VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: true } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.206951 28509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: LEARNER
I20260811 00:19:49.208362 28702 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: attempting to promote NON_VOTER fbc8732d8d19493c862e2c1519fae618 to VOTER
I20260811 00:19:49.209358 28702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } }
I20260811 00:19:49.211526 28451 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 LEARNER]: Refusing update from remote peer 2dada80505ec4b0f87382962092f83b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 00:19:49.212452 28703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 00:19:49.212785 27330 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 -> Peer 0c67b5d559af48b7bcf14cdfc1f58965 (127.16.71.188:33237): Couldn't send request to peer 0c67b5d559af48b7bcf14cdfc1f58965. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 00:19:49.214958 28365 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fbc8732d8d19493c862e2c1519fae618 (127.16.71.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.218777 28365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.219075 28365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:49.219347 28702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.219623 28702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:49.219485 28451 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fbc8732d8d19493c862e2c1519fae618 (127.16.71.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.223268 28509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.223512 28509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:19:49.231024 28374 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:49.231545 28374 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:49.231788 28374 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: stopping tablet replica
I20260811 00:19:49.238417 28374 master.cc:552] Master@127.16.71.188:33237 shutdown complete.
I20260811 00:19:49.238521 28374 master_runner.cc:305] Clearing existing system tablet
I20260811 00:19:49.238895 28374 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:49.239012 28374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:49.241534 28374 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 00:19:49.243067 28710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:49.243438 28374 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:49.243703 28374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 00:19:49.243817 28374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
uuid: "0c67b5d559af48b7bcf14cdfc1f58965"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "040ff36fcbdabc4c1bedc5661c465eb4"
server_key_iv: "42323285e59e26e391277028ada617b6"
server_key_version: "encryptionkey@0"
I20260811 00:19:49.247615 28374 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:49.255771 28374 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 00:19:49.256081 28374 log.cc:1196] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 00:19:49.256675 28374 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Deleting consensus metadata
I20260811 00:19:49.256999 28374 master_runner.cc:315] Copying system tablet from 127.16.71.190:42327
I20260811 00:19:49.260267 28374 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: tablet copy: Beginning tablet copy session from remote peer at address 127.16.71.190:42327
I20260811 00:19:49.282279 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.261306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35112 (local address 127.16.71.190:42327)
0811 00:19:49.261553 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.261560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.265401 (+ 3841us) server_negotiation.cc:407] Connection header received
0811 00:19:49.265607 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.265613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.265714 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.265845 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.267157 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.268352 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.270701 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.271382 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.271541 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.274481 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.274489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.274494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.274552 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.278637 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.279644 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.279651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.279655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.279834 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.280332 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.280338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.280341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.280808 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.280973 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.281603 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.281887 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 00:19:49.282459 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.261240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:35112)
0811 00:19:49.265206 (+ 3966us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.265237 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.265441 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.265909 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.265915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.266545 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.266852 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.266868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.268583 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.268590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.270451 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.270480 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.271602 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.271609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.271833 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.272839 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.272881 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.274172 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.278866 (+ 4694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.278878 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.278883 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.279465 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.279972 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.279977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.279981 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.280189 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.280973 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.280979 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.281204 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.281364 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.281740 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3758,"thread_start_us":3581,"threads_started":1}
I20260811 00:19:49.283324 27396 tablet_copy_service.cc:140] P 2dada80505ec4b0f87382962092f83b7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0c67b5d559af48b7bcf14cdfc1f58965 ({username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:35112)
I20260811 00:19:49.283524 27396 tablet_copy_service.cc:161] P 2dada80505ec4b0f87382962092f83b7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0c67b5d559af48b7bcf14cdfc1f58965 at {username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:35112: session id = 0c67b5d559af48b7bcf14cdfc1f58965-00000000000000000000000000000000
I20260811 00:19:49.287601 27396 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 00:19:49.289829 28374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 00:19:49.297645 28374 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: tablet copy: Starting download of 0 data blocks...
I20260811 00:19:49.298020 28374 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: tablet copy: Starting download of 1 WAL segments...
I20260811 00:19:49.301146 28374 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 00:19:49.306859 27396 tablet_copy_service.cc:342] P 2dada80505ec4b0f87382962092f83b7: Request end of tablet copy session 0c67b5d559af48b7bcf14cdfc1f58965-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:35112
I20260811 00:19:49.307112 27396 tablet_copy_service.cc:433] P 2dada80505ec4b0f87382962092f83b7: ending tablet copy session 0c67b5d559af48b7bcf14cdfc1f58965-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0c67b5d559af48b7bcf14cdfc1f58965
I20260811 00:19:49.319825 28374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:49.322777 28720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:49.327678 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 27462
W20260811 00:19:49.327701 28722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:49.341185 28374 server_base.cc:1034] running on GCE node
W20260811 00:19:49.345052 28725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:49.345655 28374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:49.346378 28374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:49.347616 28374 hybrid_clock.cc:648] HybridClock initialized: now 1786407589347576 us; error 38 us; skew 500 ppm
I20260811 00:19:49.347855 28374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407589, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:49.349825 28374 init.cc:377] Logged in from keytab as oryx/127.16.71.188@KRBTEST.COM (short username oryx)
I20260811 00:19:49.353741 28374 webserver.cc:460] Webserver started at http://127.16.71.188:37801/ using document root <none> and password file <none>
I20260811 00:19:49.354077 28374 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:49.354184 28374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:49.356930 28374 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 00:19:49.358915 28742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:49.359349 28374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 00:19:49.359489 28374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
uuid: "0c67b5d559af48b7bcf14cdfc1f58965"
format_stamp: "Formatted at 2026-08-11 00:19:48 on dist-test-slave-f7th"
server_key: "040ff36fcbdabc4c1bedc5661c465eb4"
server_key_iv: "42323285e59e26e391277028ada617b6"
server_key_version: "encryptionkey@0"
I20260811 00:19:49.359683 28374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:49.365073 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:45607
--local_ip_for_outbound_sockets=127.16.71.129
--tserver_master_addrs=127.16.71.190:42327
--webserver_port=37721
--webserver_interface=127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab
--principal=oryx/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:19:49.376338 28374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:49.379858 28374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:49.432086 28374 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.188:33237
I20260811 00:19:49.432111 28832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.188:33237 every 8 connection(s)
I20260811 00:19:49.433143 28374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
I20260811 00:19:49.438251 28833 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:19:49.441397 28833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Bootstrap starting.
I20260811 00:19:49.458498 28833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:19:49.459043 28833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Bootstrap complete.
I20260811 00:19:49.460119 28833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } }
I20260811 00:19:49.460412 28833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0c67b5d559af48b7bcf14cdfc1f58965, State: Initialized, Role: LEARNER
I20260811 00:19:49.460728 28833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } }
I20260811 00:19:49.461644 28840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.462035 28833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:49.462119 28840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: This master's current role is: LEARNER
I20260811 00:19:49.477380 28857 catalog_manager.cc:1259] Loaded cluster ID: 51a418b47455482897f68fdfa0314b89
I20260811 00:19:49.477556 28857 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: loading cluster ID for follower catalog manager: success
I20260811 00:19:49.480635 28857 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: acquiring CA information for follower catalog manager: success
I20260811 00:19:49.482735 28857 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:19:49.688189 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.680545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.0.0.1:48244)
0811 00:19:49.681160 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.681186 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.681372 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.682027 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.682031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.682060 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:49.682268 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.682277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.684192 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.684197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.685672 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.685698 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.687473 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.687479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.687637 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.687666 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.687753 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.687868 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":338,"threads_started":1}
I20260811 00:19:49.688323 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.680808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48244 (local address 127.16.71.188:33237)
0811 00:19:49.681593 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.681601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.681626 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:19:49.681713 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.681717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.681820 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.681957 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:49.682434 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.683970 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.685913 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.687244 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.687405 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.687678 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.687803 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.687956 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":453,"threads_started":1}
I20260811 00:19:49.689517 28792 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 00:19:49.690248 28840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dada80505ec4b0f87382962092f83b7. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:49.690565 28840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: This master's current role is: LEARNER
W20260811 00:19:49.802063 28750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:49.802409 28750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:49.802553 28750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:49.810365 28750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:49.810521 28750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:49.810685 28750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:49.821939 28750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab
--principal=oryx/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:45607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=37721
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:49.823467 28750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:49.825001 28750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:49.840878 28870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:49.841069 28872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:49.841300 28869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:49.841816 28750 server_base.cc:1034] running on GCE node
I20260811 00:19:49.842555 28750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:49.843875 28750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:49.845103 28750 hybrid_clock.cc:648] HybridClock initialized: now 1786407589845025 us; error 74 us; skew 500 ppm
I20260811 00:19:49.845552 28750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407589, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:49.849601 28750 init.cc:377] Logged in from keytab as oryx/127.16.71.129@KRBTEST.COM (short username oryx)
I20260811 00:19:49.851795 28750 webserver.cc:460] Webserver started at http://127.16.71.129:37721/ using document root <none> and password file <none>
I20260811 00:19:49.852782 28750 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:49.852991 28750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:49.860666 28750 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:19:49.864897 28884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:49.867276 28750 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:19:49.867548 28750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "50104b6eb2364c70a066d1547eeb9591"
format_stamp: "Formatted at 2026-08-11 00:19:46 on dist-test-slave-f7th"
server_key: "4569c7411a94b3c49e645db31ce45b5f"
server_key_iv: "853170b1e7816dd6cec9427ab3f53146"
server_key_version: "encryptionkey@0"
I20260811 00:19:49.869535 28750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:49.890260 28750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:49.894853 28750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:49.895123 28750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:49.896193 28750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:49.897768 28750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:49.897876 28750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:49.897964 28750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:49.898020 28750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:49.936568 28750 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:45607
I20260811 00:19:49.936689 29087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:45607 every 8 connection(s)
I20260811 00:19:49.938267 28750 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:49.941797 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 28750
I20260811 00:19:49.942219 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 27719
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.130@KRBTEST.COM
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407589, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407589, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.189@KRBTEST.COM
Aug 11 00:19:49 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407589, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.188@KRBTEST.COM
I20260811 00:19:49.976905 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.943998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.16.71.130:41085)
0811 00:19:49.953083 (+ 9085us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.953095 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.953142 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:19:49.953250 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.953261 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.953623 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.953878 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.955924 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.957566 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.959473 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.960340 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.961262 (+ 922us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.967237 (+ 5975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.967257 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.967278 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.967326 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.971386 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.973238 (+ 1852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.973249 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.973253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.973447 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.974004 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.974010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.974013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.974538 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.974852 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.976132 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.976422 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8857,"thread_start_us":8680,"threads_started":1}
I20260811 00:19:49.978755 27934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:49.981606 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.957983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:45851 (local address 127.16.71.188:33237)
0811 00:19:49.958182 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.958188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.959158 (+ 970us) server_negotiation.cc:407] Connection header received
0811 00:19:49.959416 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.959422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.959503 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.959573 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.962356 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.963451 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.966257 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.966942 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.967066 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.973921 (+ 6855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.973929 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.973934 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.973983 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.976883 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.978950 (+ 2067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.978958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.978961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.979143 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.979650 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.979655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.979658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.980029 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.980174 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.980937 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.981226 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:19:49.982242 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.944414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.16.71.129:45539)
0811 00:19:49.957369 (+ 12955us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.957392 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.958506 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.959514 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.959518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.960787 (+ 1269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.961718 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.961743 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.963501 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.963507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.966063 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.966080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.967231 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.967239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.967696 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.968657 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.968700 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.971731 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.978508 (+ 6777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.978514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.978517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.979116 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.979673 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.979679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.979683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.979863 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.980774 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.980780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.980968 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.981140 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.981484 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12799,"mutex_wait_us":394,"spinlock_wait_cycles":71808,"thread_start_us":9877,"threads_started":1}
I20260811 00:19:49.983290 29099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.943437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:45539 (local address 127.16.71.189:38833)
0811 00:19:49.951484 (+ 8047us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.951494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.957607 (+ 6113us) server_negotiation.cc:407] Connection header received
0811 00:19:49.959179 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.959186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.959290 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.960186 (+ 896us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.961932 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.963254 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.966263 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.966947 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.967607 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.971993 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.972000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.972005 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.972062 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.978321 (+ 6259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.979334 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.979341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.979345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.979533 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.980015 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.980020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.980024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.980480 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.982387 (+ 1907us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.982589 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.982873 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7882,"thread_start_us":7756,"threads_started":1}
I20260811 00:19:49.983289 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.943093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:54529 (local address 127.16.71.190:42327)
0811 00:19:49.944296 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:49.944304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:49.955069 (+ 10765us) server_negotiation.cc:407] Connection header received
0811 00:19:49.958442 (+ 3373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:49.958449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:49.958557 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:49.958671 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:49.961973 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.963078 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.966268 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.966931 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.967063 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.971801 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:49.971810 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:49.971815 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:49.971862 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:49.975818 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.976960 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.976967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.976970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.977228 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:49.977834 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:49.977839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:49.977842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:49.978226 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:49.978389 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:49.982171 (+ 3782us) server_negotiation.cc:299] Negotiation successful
0811 00:19:49.982718 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1030,"thread_start_us":884,"threads_started":1}
I20260811 00:19:49.983633 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.957923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.16.71.129:45851)
0811 00:19:49.959048 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.959074 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.959265 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.959790 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.959794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.960920 (+ 1126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.962168 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.962182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.963671 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.963676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.966079 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.966089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.967378 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.967384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.967687 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.970202 (+ 2515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.970227 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.973599 (+ 3372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.977035 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.977041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.977045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.978762 (+ 1717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.979302 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.979307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.979311 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.979508 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.980258 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.980262 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.980427 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.981912 (+ 1485us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.982215 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":720,"thread_start_us":835,"threads_started":1}
I20260811 00:19:49.984500 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:49.943291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:54529)
0811 00:19:49.954807 (+ 11516us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:49.954881 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:49.958024 (+ 3143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:49.959190 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:49.959205 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:49.960544 (+ 1339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:49.961713 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.961745 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.963289 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.963295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.966059 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:49.966077 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:49.967316 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:49.967322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:49.967738 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:49.968665 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:49.968726 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:49.971477 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:49.976010 (+ 4533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.976019 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.976049 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.976754 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.977388 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:49.977394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:49.977398 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:49.977675 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:49.978594 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:49.978603 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:49.979248 (+ 645us) client_negotiation.cc:769] Sending connection context
0811 00:19:49.979599 (+ 351us) client_negotiation.cc:240] Negotiation successful
0811 00:19:49.980243 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11139,"mutex_wait_us":377,"spinlock_wait_cycles":47104,"thread_start_us":13396,"threads_started":1}
I20260811 00:19:49.984899 29095 heartbeater.cc:347] Connected to a master server at 127.16.71.189:38833
I20260811 00:19:49.985219 29095 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:49.985793 29091 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42327
I20260811 00:19:49.986002 29091 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:49.986436 29095 heartbeater.cc:511] Master 127.16.71.189:38833 requested a full tablet report, sending...
I20260811 00:19:49.986479 29091 heartbeater.cc:511] Master 127.16.71.190:42327 requested a full tablet report, sending...
I20260811 00:19:49.987156 29090 heartbeater.cc:347] Connected to a master server at 127.16.71.188:33237
I20260811 00:19:49.987317 29090 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:49.987804 29090 heartbeater.cc:511] Master 127.16.71.188:33237 requested a full tablet report, sending...
I20260811 00:19:49.988471 27358 ts_manager.cc:194] Re-registered known tserver with Master: 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607)
I20260811 00:19:49.989645 27358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:54529
I20260811 00:19:49.989850 28773 ts_manager.cc:194] Registered new tserver with Master: 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607)
I20260811 00:19:49.989886 28438 ts_manager.cc:194] Registered new tserver with Master: 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607)
I20260811 00:19:50.057356 28702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: NON_VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: true } } }
I20260811 00:19:50.057741 28702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:50.060132 29109 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: attempting to promote NON_VOTER 0c67b5d559af48b7bcf14cdfc1f58965 to VOTER
I20260811 00:19:50.060956 29109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } }
I20260811 00:19:50.063105 28451 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Refusing update from remote peer 2dada80505ec4b0f87382962092f83b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 00:19:50.063509 28792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 LEARNER]: Refusing update from remote peer 2dada80505ec4b0f87382962092f83b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 00:19:50.063863 29110 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 00:19:50.064406 29109 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 00:19:50.067368 28702 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0c67b5d559af48b7bcf14cdfc1f58965 (127.16.71.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.069272 28792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0c67b5d559af48b7bcf14cdfc1f58965 (127.16.71.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.072599 28840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.072959 28840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:19:50.073031 28702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.073369 28702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:50.073225 28451 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0c67b5d559af48b7bcf14cdfc1f58965 (127.16.71.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.073266 29118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.073764 29118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:50.077183 29109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.077548 29109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:50.077431 29120 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dada80505ec4b0f87382962092f83b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:50.077847 29120 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:19:50.084182 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 27719
I20260811 00:19:50.119529 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:41085
--local_ip_for_outbound_sockets=127.16.71.130
--tserver_master_addrs=127.16.71.190:42327
--webserver_port=40965
--webserver_interface=127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab
--principal=oryx/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:50.502013 29121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:50.502370 29121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:50.502519 29121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:50.511670 29121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:50.511821 29121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:50.511943 29121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:50.523660 29121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab
--principal=oryx/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:41085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=40965
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:50.525254 29121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:50.526786 29121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:50.542095 29136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:50.542243 29129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:50.542150 29131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:50.543032 29121 server_base.cc:1034] running on GCE node
I20260811 00:19:50.543742 29121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:50.544920 29121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:50.546170 29121 hybrid_clock.cc:648] HybridClock initialized: now 1786407590546107 us; error 53 us; skew 500 ppm
I20260811 00:19:50.546540 29121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:50 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407590, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:50.550420 29121 init.cc:377] Logged in from keytab as oryx/127.16.71.130@KRBTEST.COM (short username oryx)
I20260811 00:19:50.552254 29121 webserver.cc:460] Webserver started at http://127.16.71.130:40965/ using document root <none> and password file <none>
I20260811 00:19:50.553128 29121 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:50.553289 29121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:50.560016 29121 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:50.563563 29150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:50.565503 29121 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:50.565690 29121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "23c41039d6264036a6970bab6d2280f4"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "569d5cb6b133a97f5e31b2e8c574b9b8"
server_key_iv: "10fb75e77455598509d759715ed7728e"
server_key_version: "encryptionkey@0"
I20260811 00:19:50.566830 29121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:50.592752 29121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:50.597582 29121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:50.597893 29121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:50.599033 29121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:50.600934 29121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:50.601044 29121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:50.601158 29121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:50.601230 29121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:50.636803 29121 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:41085
I20260811 00:19:50.636875 29340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:41085 every 8 connection(s)
I20260811 00:19:50.638473 29121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:50.639163 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 29121
I20260811 00:19:50.639463 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 27952
Aug 11 00:19:50 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407590, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.189@KRBTEST.COM
Aug 11 00:19:50 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407590, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
Aug 11 00:19:50 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407590, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.188@KRBTEST.COM
Aug 11 00:19:50 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.131@KRBTEST.COM
I20260811 00:19:50.676900 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:50.644254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:51997 (local address 127.16.71.189:38833)
0811 00:19:50.654249 (+ 9995us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:50.654254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:50.654271 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:19:50.654330 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:50.654333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:50.654402 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:50.654524 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:50.657347 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.658625 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.661770 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.662461 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.662639 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:50.668119 (+ 5480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:50.668127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:50.668132 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:50.668181 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:50.671965 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.673104 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.673111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.673115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.673341 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.673870 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.673876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.673880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.674322 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:50.674500 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:50.676131 (+ 1631us) server_negotiation.cc:299] Negotiation successful
0811 00:19:50.676456 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9891,"thread_start_us":9783,"threads_started":1}
I20260811 00:19:50.678846 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:50.647015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:42123 (local address 127.16.71.190:42327)
0811 00:19:50.649212 (+ 2197us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:50.649221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:50.655290 (+ 6069us) server_negotiation.cc:407] Connection header received
0811 00:19:50.655598 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:50.655604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:50.655689 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:50.655769 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:50.657908 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.658996 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.663174 (+ 4178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.664197 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.664311 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:50.670609 (+ 6298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:50.670617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:50.670622 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:50.670670 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:50.674736 (+ 4066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.676097 (+ 1361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.676104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.676108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.676296 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.676962 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.676967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.676971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.677391 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:50.677546 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:50.678160 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 00:19:50.678452 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":1827,"threads_started":1}
I20260811 00:19:50.679464 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:50.643476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.16.71.130:51997)
0811 00:19:50.650008 (+ 6532us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:50.650095 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:50.652961 (+ 2866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:50.654931 (+ 1970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:50.654947 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:50.655883 (+ 936us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:50.657069 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:50.657102 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.658899 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.658905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:50.661438 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:50.661458 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.662797 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.662807 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:50.663274 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:50.664109 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:50.664143 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:50.667754 (+ 3611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:50.672162 (+ 4408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:50.672174 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:50.672198 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:50.672912 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:50.673476 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:50.673481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:50.673486 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:50.673721 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:50.674603 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:50.674612 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:50.675271 (+ 659us) client_negotiation.cc:769] Sending connection context
0811 00:19:50.675645 (+ 374us) client_negotiation.cc:240] Negotiation successful
0811 00:19:50.677737 (+ 2092us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6084,"mutex_wait_us":46,"thread_start_us":11155,"threads_started":1}
I20260811 00:19:50.680007 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:50.647035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.130:42123)
0811 00:19:50.655163 (+ 8128us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:50.655183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:50.655416 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:50.655991 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:50.655995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:50.656658 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:50.657707 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:50.657720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.659207 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.659212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:50.662950 (+ 3738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:50.662974 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.664750 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.664759 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:50.664968 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:50.665793 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:50.665820 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:50.670324 (+ 4504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:50.675049 (+ 4725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:50.675055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:50.675059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:50.675798 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:50.676611 (+ 813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:50.676616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:50.676620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:50.676810 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:50.677641 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:50.677645 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:50.677847 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 00:19:50.679168 (+ 1321us) client_negotiation.cc:240] Negotiation successful
0811 00:19:50.679543 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7951,"mutex_wait_us":1248,"thread_start_us":2392,"threads_started":1}
I20260811 00:19:50.681612 29343 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42327
I20260811 00:19:50.681643 29344 heartbeater.cc:347] Connected to a master server at 127.16.71.189:38833
I20260811 00:19:50.682052 29344 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:50.682056 29343 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:50.683382 29344 heartbeater.cc:511] Master 127.16.71.189:38833 requested a full tablet report, sending...
I20260811 00:19:50.683411 29343 heartbeater.cc:511] Master 127.16.71.190:42327 requested a full tablet report, sending...
I20260811 00:19:50.684475 29356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:50.651042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:49843 (local address 127.16.71.188:33237)
0811 00:19:50.659963 (+ 8921us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:50.659969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:50.660581 (+ 612us) server_negotiation.cc:407] Connection header received
0811 00:19:50.660742 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:50.660747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:50.660836 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:50.660947 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:50.666668 (+ 5721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.667922 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.669892 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.670540 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.670677 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:50.675013 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:50.675021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:50.675026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:50.675074 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:50.680422 (+ 5348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.681413 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.681420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.681424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.681613 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.682347 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.682353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.682356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.682778 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:50.683033 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:50.683747 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 00:19:50.684069 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8772,"thread_start_us":8666,"threads_started":1}
I20260811 00:19:50.685637 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:50.652417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.16.71.130:49843)
0811 00:19:50.657287 (+ 4870us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:50.657306 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:50.657407 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:50.665689 (+ 8282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:50.665695 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:50.666180 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:50.666470 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:50.666481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.668129 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.668134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:50.669669 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:50.669686 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.670941 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.670962 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:50.671153 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:50.671795 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:50.671820 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:50.674741 (+ 2921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:50.680602 (+ 5861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:50.680607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:50.680611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:50.681241 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:50.681842 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:50.681847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:50.681850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:50.682070 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:50.683120 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:50.683126 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:50.683328 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 00:19:50.684820 (+ 1492us) client_negotiation.cc:240] Negotiation successful
0811 00:19:50.685238 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4744,"thread_start_us":4568,"threads_started":1}
I20260811 00:19:50.685674 27358 ts_manager.cc:194] Re-registered known tserver with Master: 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085)
I20260811 00:19:50.685843 28438 ts_manager.cc:194] Registered new tserver with Master: 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085)
I20260811 00:19:50.687018 27358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:42123
I20260811 00:19:50.687104 29341 heartbeater.cc:347] Connected to a master server at 127.16.71.188:33237
I20260811 00:19:50.687250 29341 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:50.688043 29341 heartbeater.cc:511] Master 127.16.71.188:33237 requested a full tablet report, sending...
I20260811 00:19:50.688110 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:50.640815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.16.71.131:33589)
0811 00:19:50.653951 (+ 13136us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:50.653960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:50.660486 (+ 6526us) server_negotiation.cc:407] Connection header received
0811 00:19:50.660755 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:50.660766 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:50.661132 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:50.661299 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:50.663668 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.665346 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.667352 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:50.668461 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:50.668602 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:50.677896 (+ 9294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:50.677922 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:50.677942 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:50.678002 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:50.682843 (+ 4841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.684021 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.684034 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.684037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.684247 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:50.684828 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:50.684834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:50.684837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:50.685375 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:50.685715 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:50.687241 (+ 1526us) server_negotiation.cc:299] Negotiation successful
0811 00:19:50.687774 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12965,"thread_start_us":12785,"threads_started":1}
I20260811 00:19:50.689742 28773 ts_manager.cc:194] Registered new tserver with Master: 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085)
I20260811 00:19:50.690411 28166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:50.776773 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 27952
I20260811 00:19:50.799769 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:33589
--local_ip_for_outbound_sockets=127.16.71.131
--tserver_master_addrs=127.16.71.190:42327
--webserver_port=40265
--webserver_interface=127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab
--principal=oryx/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:19:50.993054 29091 heartbeater.cc:503] Master 127.16.71.190:42327 was elected leader, sending a full tablet report...
W20260811 00:19:51.185673 29360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:51.186015 29360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:51.186113 29360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:51.194028 29360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:51.194164 29360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:51.194314 29360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:51.205533 29360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab
--principal=oryx/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:33589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=40265
--tserver_master_addrs=127.16.71.190:42327,127.16.71.189:38833,127.16.71.188:33237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:51.207219 29360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:51.208720 29360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:51.223960 29371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:51.224005 29373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:51.224043 29367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:51.224674 29360 server_base.cc:1034] running on GCE node
I20260811 00:19:51.225410 29360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:51.226518 29360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:51.227744 29360 hybrid_clock.cc:648] HybridClock initialized: now 1786407591227680 us; error 65 us; skew 500 ppm
I20260811 00:19:51.228116 29360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:51 dist-test-slave-f7th krb5kdc[27269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407591, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:51.235816 29360 init.cc:377] Logged in from keytab as oryx/127.16.71.131@KRBTEST.COM (short username oryx)
I20260811 00:19:51.237916 29360 webserver.cc:460] Webserver started at http://127.16.71.131:40265/ using document root <none> and password file <none>
I20260811 00:19:51.238729 29360 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:51.238898 29360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:51.245213 29360 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:51.249539 29389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:51.251793 29360 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260811 00:19:51.252019 29360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "9d6fd1186d724dd78559db05d7d42c3a"
format_stamp: "Formatted at 2026-08-11 00:19:47 on dist-test-slave-f7th"
server_key: "44990e12cd855c506076960f972e9abd"
server_key_iv: "7739758f8ef9c5fa33bad881daef3c7e"
server_key_version: "encryptionkey@0"
I20260811 00:19:51.253574 29360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:51.292972 29360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:51.297662 29360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:51.297983 29360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:51.299171 29360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:19:51.300793 29360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:51.300895 29360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:51.301021 29360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:51.301105 29360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:51.336488 29360 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:33589
I20260811 00:19:51.336571 29569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:33589 every 8 connection(s)
I20260811 00:19:51.338281 29360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:51.340242 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 29360
Aug 11 00:19:51 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407591, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.189@KRBTEST.COM
I20260811 00:19:51.366742 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.342825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:56299 (local address 127.16.71.189:38833)
0811 00:19:51.343336 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.343342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.344367 (+ 1025us) server_negotiation.cc:407] Connection header received
0811 00:19:51.347437 (+ 3070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.347443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.347543 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.347647 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.349578 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.350608 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.352890 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.353461 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.353595 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.357746 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.357754 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.357760 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.357807 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.361183 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.362226 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.362233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.362237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.362405 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.362992 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.363000 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.363004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.363349 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.363502 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.364557 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.364865 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":635,"threads_started":1}
I20260811 00:19:51.367233 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.342172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.16.71.131:56299)
0811 00:19:51.344137 (+ 1965us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.344212 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.347092 (+ 2880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.347955 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.347966 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.348656 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:51.349356 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.349377 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.350827 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.350831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.352674 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.352696 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.353745 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.353754 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.354017 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.354756 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:51.354783 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:51.357385 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:51.361395 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:51.361405 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:51.361422 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:51.362059 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:51.362543 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:51.362547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:51.362549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:51.362788 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:51.363509 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:51.363518 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:51.364051 (+ 533us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.364442 (+ 391us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.365200 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1583,"thread_start_us":20864,"threads_started":1}
I20260811 00:19:51.369304 29572 heartbeater.cc:347] Connected to a master server at 127.16.71.189:38833
I20260811 00:19:51.369771 29572 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:51.370967 29572 heartbeater.cc:511] Master 127.16.71.189:38833 requested a full tablet report, sending...
Aug 11 00:19:51 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407591, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.188@KRBTEST.COM
Aug 11 00:19:51 dist-test-slave-f7th krb5kdc[27269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407591, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:51.377993 28438 ts_manager.cc:194] Registered new tserver with Master: 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
I20260811 00:19:51.387032 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.343247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.131:51173)
0811 00:19:51.362666 (+ 19419us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.362693 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.362880 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.364008 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.364014 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.364865 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:51.365187 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.365202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.366665 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.366671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.368335 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.368357 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.372199 (+ 3842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.372207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.372461 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.374089 (+ 1628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:51.374115 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:51.377708 (+ 3593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:51.383151 (+ 5443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:51.383159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:51.383164 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:51.384068 (+ 904us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:51.384748 (+ 680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:51.384754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:51.384758 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:51.384988 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:51.385777 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:51.385783 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:51.386018 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.386164 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.386539 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19110,"thread_start_us":366,"threads_started":1}
I20260811 00:19:51.388490 29579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.342008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:51173 (local address 127.16.71.190:42327)
0811 00:19:51.342783 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.342791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.362833 (+ 20042us) server_negotiation.cc:407] Connection header received
0811 00:19:51.363667 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.363672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.363764 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.363875 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.365376 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.366498 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.368537 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.369881 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.370000 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.377997 (+ 7997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.378006 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.378011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.378065 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.382911 (+ 4846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.384277 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.384284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.384288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.384540 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.385180 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.385186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.385190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.385608 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.387527 (+ 1919us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.387690 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.387981 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":479,"threads_started":1}
I20260811 00:19:51.389458 29571 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42327
I20260811 00:19:51.390108 29571 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:51.390815 29571 heartbeater.cc:511] Master 127.16.71.190:42327 requested a full tablet report, sending...
I20260811 00:19:51.392119 27358 ts_manager.cc:194] Re-registered known tserver with Master: 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
I20260811 00:19:51.392330 29590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.345464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:55397 (local address 127.16.71.188:33237)
0811 00:19:51.361797 (+ 16333us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.361805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.363718 (+ 1913us) server_negotiation.cc:407] Connection header received
0811 00:19:51.363813 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.363818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.363904 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.364010 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.365930 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.367147 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.369798 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.370417 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.370524 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.378780 (+ 8256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.378788 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.378793 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.378842 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.382853 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.389611 (+ 6758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.389617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.389621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.389834 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.390485 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.390491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.390494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.390895 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.391075 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.391663 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.391953 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16171,"thread_start_us":16383,"threads_started":1}
I20260811 00:19:51.393376 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.345004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.16.71.131:55397)
0811 00:19:51.363456 (+ 18452us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.363474 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.363609 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.364212 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.364217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.365285 (+ 1068us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:51.365709 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.365724 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.367315 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.367320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.369589 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.369608 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.370987 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.370995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.371201 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.372157 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:51.372190 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:51.375930 (+ 3740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:51.383228 (+ 7298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:51.383240 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:51.383244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:51.389253 (+ 6009us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:51.390038 (+ 785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:51.390042 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:51.390046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:51.390328 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:51.391169 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:51.391173 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:51.391355 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.392529 (+ 1174us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.392936 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18314,"thread_start_us":17132,"threads_started":1}
I20260811 00:19:51.393388 27358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:51173
I20260811 00:19:51.399199 29570 heartbeater.cc:347] Connected to a master server at 127.16.71.188:33237
I20260811 00:19:51.399417 29570 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:51.399931 29570 heartbeater.cc:511] Master 127.16.71.188:33237 requested a full tablet report, sending...
I20260811 00:19:51.401515 28773 ts_manager.cc:194] Registered new tserver with Master: 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
I20260811 00:19:51.404322 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.370881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.16.71.190:42327)
0811 00:19:51.379352 (+ 8471us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.379360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.379388 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:19:51.379462 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.379482 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.379580 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.379718 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.382496 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.384506 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.387668 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.388407 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.388564 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.392866 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.392873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.392878 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.392925 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.397032 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.398532 (+ 1500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.398538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.398541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.398777 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.400738 (+ 1961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.400744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.400747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.401190 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.401480 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.403670 (+ 2190us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.403953 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7357,"thread_start_us":9389,"threads_started":1}
I20260811 00:19:51.412357 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.383141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.16.71.188:33237)
0811 00:19:51.389790 (+ 6649us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.389797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.389819 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:19:51.389892 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.389897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.389979 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.390100 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.393921 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.395189 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.398000 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.398522 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.398668 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.403298 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.403305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.403309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.403358 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.407813 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.408889 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.408896 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.408900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.409091 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.409600 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.409604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.409607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.409922 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.410177 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.411256 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.411844 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6496,"thread_start_us":8306,"threads_started":1}
I20260811 00:19:51.414077 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.374149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.16.71.189:38833)
0811 00:19:51.374807 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.374815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.380694 (+ 5879us) server_negotiation.cc:407] Connection header received
0811 00:19:51.380854 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.380859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.380950 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.381065 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.386507 (+ 5442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.387955 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.395225 (+ 7270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.395910 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.398527 (+ 2617us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.403264 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.403274 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.403279 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.403329 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.408858 (+ 5529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.409749 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.409756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.409759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.409949 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.410482 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.410488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.410491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.410929 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.411581 (+ 652us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.413576 (+ 1995us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.413807 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461}
I20260811 00:19:51.410370 27358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:51.419023 27358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:51.473446 29624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.448653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:45607 (local address 127.0.0.1:39404)
0811 00:19:51.449464 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.449488 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.458050 (+ 8562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.465728 (+ 7678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.465734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.465763 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:51.466083 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.466094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.467865 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.467870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.470042 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.470058 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.472538 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.472544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.472759 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.472797 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.472904 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.473075 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":470,"threads_started":1}
I20260811 00:19:51.474344 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.449984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:41085 (local address 127.0.0.1:58150)
0811 00:19:51.465206 (+ 15222us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.465223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.465368 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.467178 (+ 1810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.467182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.467210 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:51.467622 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.467637 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.469288 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.469294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.471586 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.471603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.473527 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.473535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.473718 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.473746 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.473839 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.473990 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14413,"thread_start_us":14741,"threads_started":1}
I20260811 00:19:51.474376 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.450306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.16.71.130:41085)
0811 00:19:51.466293 (+ 15987us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.466303 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.466334 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:19:51.466450 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.466458 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.466681 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.466932 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:51.467840 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.469088 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.471787 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.473234 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.473394 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.473640 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.473924 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.474119 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15806,"thread_start_us":16534,"threads_started":1}
I20260811 00:19:51.474406 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.448936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.16.71.129:45607)
0811 00:19:51.464371 (+ 15435us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.464380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.464418 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:19:51.464556 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.464565 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.465296 (+ 731us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.465596 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:51.466283 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.467686 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.470236 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.471950 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.472102 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.472364 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.473871 (+ 1507us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.474083 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15229,"thread_start_us":16027,"threads_started":1}
I20260811 00:19:51.479108 28975 tablet_service.cc:1467] Processing CreateTablet for tablet 0a200ddc5df24519ae673980f0bc4daa (DEFAULT_TABLE table=test-table [id=a130ae48a3dd4cd8935bd0e369a84225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:51.479108 29235 tablet_service.cc:1467] Processing CreateTablet for tablet 0a200ddc5df24519ae673980f0bc4daa (DEFAULT_TABLE table=test-table [id=a130ae48a3dd4cd8935bd0e369a84225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:51.480966 29235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a200ddc5df24519ae673980f0bc4daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:51.481086 28975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a200ddc5df24519ae673980f0bc4daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:51.485823 29627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.450366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:33589 (local address 127.0.0.1:53690)
0811 00:19:51.465163 (+ 14797us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.465191 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.465392 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.466504 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.466509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.466538 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:51.466892 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.466906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.470228 (+ 3322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.470232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.472329 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.472347 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.484732 (+ 12385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.484741 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.485018 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.485062 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.485216 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.485382 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14638,"mutex_wait_us":3,"thread_start_us":13863,"threads_started":1}
I20260811 00:19:51.485909 29633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.451038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.16.71.131:33589)
0811 00:19:51.463989 (+ 12951us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.464001 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.465407 (+ 1406us) server_negotiation.cc:407] Connection header received
0811 00:19:51.465550 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.465559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.466263 (+ 704us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.466512 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:51.467166 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.469302 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.472590 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.475113 (+ 2523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.475276 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.475515 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.485332 (+ 9817us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.485595 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12698,"thread_start_us":12460,"threads_started":1}
I20260811 00:19:51.490984 29475 tablet_service.cc:1467] Processing CreateTablet for tablet 0a200ddc5df24519ae673980f0bc4daa (DEFAULT_TABLE table=test-table [id=a130ae48a3dd4cd8935bd0e369a84225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:51.492923 29475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a200ddc5df24519ae673980f0bc4daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:51.496469 29634 tablet_bootstrap.cc:492] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Bootstrap starting.
I20260811 00:19:51.499497 29635 tablet_bootstrap.cc:492] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Bootstrap starting.
I20260811 00:19:51.501499 29634 tablet_bootstrap.cc:654] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:51.502920 29634 log.cc:824] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:51.503360 29635 tablet_bootstrap.cc:654] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:51.504658 29635 log.cc:824] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:51.506978 29634 tablet_bootstrap.cc:492] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: No bootstrap required, opened a new log
I20260811 00:19:51.507417 29634 ts_tablet_manager.cc:1397] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 00:19:51.508677 29640 tablet_bootstrap.cc:492] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Bootstrap starting.
I20260811 00:19:51.508734 29635 tablet_bootstrap.cc:492] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: No bootstrap required, opened a new log
I20260811 00:19:51.509045 29635 ts_tablet_manager.cc:1397] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 00:19:51.512779 29640 tablet_bootstrap.cc:654] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:51.513936 29640 log.cc:824] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:51.517563 29640 tablet_bootstrap.cc:492] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: No bootstrap required, opened a new log
I20260811 00:19:51.517885 29640 ts_tablet_manager.cc:1397] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260811 00:19:51.519001 29634 raft_consensus.cc:348] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.519606 29634 raft_consensus.cc:374] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:51.519737 29634 raft_consensus.cc:729] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c41039d6264036a6970bab6d2280f4, State: Initialized, Role: FOLLOWER
I20260811 00:19:51.520323 29635 raft_consensus.cc:348] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.520537 29634 consensus_queue.cc:260] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.520889 29635 raft_consensus.cc:374] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:51.520996 29635 raft_consensus.cc:729] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50104b6eb2364c70a066d1547eeb9591, State: Initialized, Role: FOLLOWER
I20260811 00:19:51.521806 29635 consensus_queue.cc:260] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.524115 29635 ts_tablet_manager.cc:1428] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 00:19:51.528515 29640 raft_consensus.cc:348] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.529100 29640 raft_consensus.cc:374] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:51.529225 29640 raft_consensus.cc:729] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6fd1186d724dd78559db05d7d42c3a, State: Initialized, Role: FOLLOWER
I20260811 00:19:51.530136 29640 consensus_queue.cc:260] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.532883 29634 ts_tablet_manager.cc:1428] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Time spent starting tablet: real 0.025s user 0.024s sys 0.004s
I20260811 00:19:51.533232 29343 heartbeater.cc:503] Master 127.16.71.190:42327 was elected leader, sending a full tablet report...
I20260811 00:19:51.539000 29640 ts_tablet_manager.cc:1428] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Time spent starting tablet: real 0.021s user 0.011s sys 0.006s
I20260811 00:19:51.539670 29571 heartbeater.cc:503] Master 127.16.71.190:42327 was elected leader, sending a full tablet report...
W20260811 00:19:51.596802 29576 tablet.cc:2369] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:19:51.648929 29346 tablet.cc:2369] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:51.682588 29646 raft_consensus.cc:482] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:51.682919 29646 raft_consensus.cc:504] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.684743 29646 leader_election.cc:288] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085), 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
I20260811 00:19:51.692435 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.685271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:48419 (local address 127.16.71.130:41085)
0811 00:19:51.685485 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.685492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.686413 (+ 921us) server_negotiation.cc:407] Connection header received
0811 00:19:51.686665 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.686670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.686737 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.686838 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:51.687475 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.688206 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.690537 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.691361 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.691554 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.691641 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.691998 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.692147 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:19:51.692569 29650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.685182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:41085 (local address 127.16.71.129:48419)
0811 00:19:51.686287 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.686305 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.686481 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.686863 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.686867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.686905 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:51.687314 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.687326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.688370 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.688376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.690329 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.690348 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.691591 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.691598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.691779 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.691820 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.691934 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.692112 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":124,"thread_start_us":725,"threads_started":1}
I20260811 00:19:51.692601 29633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.685468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:43107 (local address 127.16.71.131:33589)
0811 00:19:51.685679 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.685685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.686491 (+ 806us) server_negotiation.cc:407] Connection header received
0811 00:19:51.686679 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.686684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.686749 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.686908 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:51.687538 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.688425 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.690648 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.691509 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.691680 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.691757 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.692216 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.692369 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 00:19:51.693279 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.685262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:33589 (local address 127.16.71.129:43107)
0811 00:19:51.686378 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:51.686395 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:51.686542 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:51.686925 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:51.686930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:51.686955 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:51.687358 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.687368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.688643 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.688650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.690435 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:51.690450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.691816 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.691822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:51.692027 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.692057 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:19:51.692801 (+ 744us) client_negotiation.cc:240] Negotiation successful
0811 00:19:51.692933 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":24,"thread_start_us":430,"threads_started":1}
I20260811 00:19:51.694100 29267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a200ddc5df24519ae673980f0bc4daa" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c41039d6264036a6970bab6d2280f4" is_pre_election: true
I20260811 00:19:51.694250 29507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a200ddc5df24519ae673980f0bc4daa" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6fd1186d724dd78559db05d7d42c3a" is_pre_election: true
I20260811 00:19:51.694625 29267 raft_consensus.cc:2454] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 0.
I20260811 00:19:51.694751 29507 raft_consensus.cc:2454] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 0.
I20260811 00:19:51.695386 28897 leader_election.cc:302] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 23c41039d6264036a6970bab6d2280f4, 50104b6eb2364c70a066d1547eeb9591, 9d6fd1186d724dd78559db05d7d42c3a; no voters:
I20260811 00:19:51.695837 29646 raft_consensus.cc:2781] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:51.696033 29646 raft_consensus.cc:482] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:51.696146 29646 raft_consensus.cc:3037] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:51.698460 29646 raft_consensus.cc:504] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.699128 29646 leader_election.cc:288] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 election: Requested vote from peers 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085), 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
W20260811 00:19:51.699514 29096 tablet.cc:2369] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:51.699774 29267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a200ddc5df24519ae673980f0bc4daa" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c41039d6264036a6970bab6d2280f4"
I20260811 00:19:51.699812 29507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a200ddc5df24519ae673980f0bc4daa" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6fd1186d724dd78559db05d7d42c3a"
I20260811 00:19:51.700028 29507 raft_consensus.cc:3037] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:51.700033 29267 raft_consensus.cc:3037] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:51.702744 29267 raft_consensus.cc:2454] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 1.
I20260811 00:19:51.702855 29507 raft_consensus.cc:2454] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 1.
I20260811 00:19:51.703358 28897 leader_election.cc:302] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c41039d6264036a6970bab6d2280f4, 50104b6eb2364c70a066d1547eeb9591; no voters:
I20260811 00:19:51.703756 29646 raft_consensus.cc:2781] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:51.704236 29646 raft_consensus.cc:686] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 1 LEADER]: Becoming Leader. State: Replica: 50104b6eb2364c70a066d1547eeb9591, State: Running, Role: LEADER
I20260811 00:19:51.704929 29646 consensus_queue.cc:237] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:51.710675 27356 catalog_manager.cc:5495] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "50104b6eb2364c70a066d1547eeb9591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:51.765019 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.758540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.16.71.129:45607)
0811 00:19:51.758784 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.758791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.758943 (+ 152us) server_negotiation.cc:407] Connection header received
0811 00:19:51.759219 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.759224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.759339 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.759481 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:51.759976 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.760779 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.762439 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.762910 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.763145 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.763716 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:51.764277 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:51.764475 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.764645 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.764746 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 00:19:51.774343 29267 raft_consensus.cc:1261] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Refusing update from remote peer 50104b6eb2364c70a066d1547eeb9591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:51.774374 29507 raft_consensus.cc:1261] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Refusing update from remote peer 50104b6eb2364c70a066d1547eeb9591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:51.775745 29646 consensus_queue.cc:1035] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:51.776180 29654 consensus_queue.cc:1035] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:51.789992 29662 mvcc.cc:205] Tried to move back new op lower bound from 7317125495895097344 to 7317125495626625024. Current Snapshot: MvccSnapshot[applied={T|T < 7317125495895097344}]
I20260811 00:19:51.814414 27358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:51.814774 27358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:51.822643 27358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2dada80505ec4b0f87382962092f83b7: Sending DeleteTablet for 3 replicas of tablet 0a200ddc5df24519ae673980f0bc4daa
I20260811 00:19:51.824054 28975 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a200ddc5df24519ae673980f0bc4daa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:51 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:39404
I20260811 00:19:51.824350 29235 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a200ddc5df24519ae673980f0bc4daa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:51 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:58150
I20260811 00:19:51.824397 29475 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a200ddc5df24519ae673980f0bc4daa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:51 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:53690
I20260811 00:19:51.845350 29685 tablet_replica.cc:330] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: stopping tablet replica
I20260811 00:19:51.845944 29685 raft_consensus.cc:2229] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:51.846611 29685 raft_consensus.cc:2258] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:51.847775 29687 tablet_replica.cc:330] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: stopping tablet replica
I20260811 00:19:51.847852 29682 tablet_replica.cc:330] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: stopping tablet replica
I20260811 00:19:51.848510 29682 raft_consensus.cc:2229] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:51.848492 29687 raft_consensus.cc:2229] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:51.848912 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.826801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.16.71.189:38833)
0811 00:19:51.826990 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.826996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.827016 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:19:51.827591 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.827597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.827682 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.827780 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.830224 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.831138 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.833880 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.834556 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.834685 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.840060 (+ 5375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.840069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.840074 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.840133 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.844467 (+ 4334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.845717 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.845724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.845727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.845923 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.846480 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.846485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.846488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.846984 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.847273 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.848025 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.848306 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:19:51.849185 29687 raft_consensus.cc:2258] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:51.849722 29682 raft_consensus.cc:2258] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:51.851308 29685 ts_tablet_manager.cc:1905] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:51.851374 29687 ts_tablet_manager.cc:1905] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:51.853633 29682 ts_tablet_manager.cc:1905] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:51.856202 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 27290
I20260811 00:19:51.859465 29687 ts_tablet_manager.cc:1918] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:51.859781 29687 log.cc:1196] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/0a200ddc5df24519ae673980f0bc4daa
I20260811 00:19:51.860499 29687 ts_tablet_manager.cc:1939] T 0a200ddc5df24519ae673980f0bc4daa P 9d6fd1186d724dd78559db05d7d42c3a: Deleting consensus metadata
I20260811 00:19:51.862931 27323 catalog_manager.cc:4841] TS 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589): tablet 0a200ddc5df24519ae673980f0bc4daa (table test-table [id=a130ae48a3dd4cd8935bd0e369a84225]) successfully deleted
I20260811 00:19:51.864598 29682 ts_tablet_manager.cc:1918] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:51.864884 29682 log.cc:1196] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/0a200ddc5df24519ae673980f0bc4daa
I20260811 00:19:51.864982 27410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:51.865494 29682 ts_tablet_manager.cc:1939] T 0a200ddc5df24519ae673980f0bc4daa P 50104b6eb2364c70a066d1547eeb9591: Deleting consensus metadata
I20260811 00:19:51.872965 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:51.827558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46598 (local address 127.16.71.188:33237)
0811 00:19:51.827740 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:51.827746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:51.852095 (+ 24349us) server_negotiation.cc:407] Connection header received
0811 00:19:51.852410 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:51.852415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:51.852541 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:51.852658 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:51.857002 (+ 4344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.858144 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.860659 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:51.861408 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:51.861519 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:51.865920 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:51.865928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:51.865933 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:51.865984 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:51.869711 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.870618 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.870625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.870629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.870778 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:51.871229 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:51.871235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:51.871239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:51.871566 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:51.871807 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:51.872328 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 00:19:51.872573 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:19:51.880491 29685 ts_tablet_manager.cc:1918] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:51.880793 29685 log.cc:1196] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/0a200ddc5df24519ae673980f0bc4daa
I20260811 00:19:51.881292 29685 ts_tablet_manager.cc:1939] T 0a200ddc5df24519ae673980f0bc4daa P 23c41039d6264036a6970bab6d2280f4: Deleting consensus metadata
I20260811 00:19:52.040431 27323 catalog_manager.cc:4841] TS 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607): tablet 0a200ddc5df24519ae673980f0bc4daa (table test-table [id=a130ae48a3dd4cd8935bd0e369a84225]) successfully deleted
I20260811 00:19:52.041658 27334 catalog_manager.cc:4841] TS 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085): tablet 0a200ddc5df24519ae673980f0bc4daa (table test-table [id=a130ae48a3dd4cd8935bd0e369a84225]) successfully deleted
I20260811 00:19:52.042516 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 27290
I20260811 00:19:52.103168 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:52.077705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.16.71.189:38833)
0811 00:19:52.077988 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:52.077995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:52.079671 (+ 1676us) server_negotiation.cc:407] Connection header received
0811 00:19:52.080013 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:52.080020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:52.080119 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:52.080202 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:52.083273 (+ 3071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:52.084575 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:52.087442 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:52.087915 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:52.088038 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:52.095854 (+ 7816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:52.095860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:52.095864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:52.095911 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:52.099467 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:52.100550 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:52.100556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:52.100559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:52.100779 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:52.101188 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:52.101194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:52.101198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:52.101628 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:52.101905 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:52.102334 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 00:19:52.102600 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:19:52.103636 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:52.078757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.16.71.188:33237)
0811 00:19:52.078976 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:52.078983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:52.079938 (+ 955us) server_negotiation.cc:407] Connection header received
0811 00:19:52.080209 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:52.080216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:52.080303 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:52.080379 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:52.083169 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:52.084352 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:52.087082 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:52.087681 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:52.087791 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:52.096524 (+ 8733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:52.096533 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:52.096538 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:52.096598 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:52.099894 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:52.100747 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:52.100754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:52.100757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:52.100985 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:52.101424 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:52.101429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:52.101432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:52.101763 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:52.102011 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:52.103070 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0811 00:19:52.103270 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:19:52.808679 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:52.807356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:41605)
0811 00:19:52.808194 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:52.808373 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":592,"threads_started":1}
I20260811 00:19:52.808933 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:52.807347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.131:34983)
0811 00:19:52.808529 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:52.808696 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":933,"threads_started":1}
I20260811 00:19:52.809024 29703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:52.808075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.130:49457)
0811 00:19:52.808683 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:52.808816 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":376,"threads_started":1}
W20260811 00:19:52.809295 29091 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:42327 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
W20260811 00:19:52.809468 29571 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:42327 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
W20260811 00:19:52.809515 29343 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:42327 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
I20260811 00:19:53.323151 29704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2dada80505ec4b0f87382962092f83b7)
I20260811 00:19:53.323458 29706 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2dada80505ec4b0f87382962092f83b7)
I20260811 00:19:53.323483 29704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } }
I20260811 00:19:53.323736 29706 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } }
I20260811 00:19:53.325157 29706 leader_election.cc:288] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2dada80505ec4b0f87382962092f83b7 (127.16.71.190:42327), 0c67b5d559af48b7bcf14cdfc1f58965 (127.16.71.188:33237)
I20260811 00:19:53.331307 29704 leader_election.cc:288] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2dada80505ec4b0f87382962092f83b7 (127.16.71.190:42327), fbc8732d8d19493c862e2c1519fae618 (127.16.71.189:38833)
I20260811 00:19:53.333040 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.325336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42804)
0811 00:19:53.332163 (+ 6827us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.332490 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6645,"thread_start_us":6375,"threads_started":1}
I20260811 00:19:53.333758 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.325601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42814)
0811 00:19:53.333235 (+ 7634us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.333472 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7495,"thread_start_us":7297,"threads_started":1}
W20260811 00:19:53.333612 28746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
W20260811 00:19:53.335659 28414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
I20260811 00:19:53.336727 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.336039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42820)
0811 00:19:53.336280 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.336383 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260811 00:19:53.337278 28746 leader_election.cc:334] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2dada80505ec4b0f87382962092f83b7 (127.16.71.190:42327): Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
I20260811 00:19:53.341917 29714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.341015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42822)
0811 00:19:53.341398 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.341634 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260811 00:19:53.342342 28414 leader_election.cc:334] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2dada80505ec4b0f87382962092f83b7 (127.16.71.190:42327): Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
I20260811 00:19:53.342594 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.325535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:50558)
0811 00:19:53.332410 (+ 6875us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.332438 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:53.332644 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:53.333313 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:53.333319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:53.333356 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:53.333844 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.333863 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.335731 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.335742 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.339147 (+ 3405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.339169 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.341666 (+ 2497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.341673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.341870 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.341914 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:19:53.341998 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 00:19:53.342151 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6723,"mutex_wait_us":94,"thread_start_us":6647,"threads_started":1}
I20260811 00:19:53.343084 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.325592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50558 (local address 127.16.71.189:38833)
0811 00:19:53.332903 (+ 7311us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.332909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.332931 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:19:53.333018 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.333022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.333118 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.334072 (+ 954us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:53.334300 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.335517 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.339431 (+ 3914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.341411 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.342347 (+ 936us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.342458 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.342518 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.342668 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":7143,"thread_start_us":6992,"threads_started":1}
I20260811 00:19:53.343798 28451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fbc8732d8d19493c862e2c1519fae618" is_pre_election: true
I20260811 00:19:53.344206 28451 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c67b5d559af48b7bcf14cdfc1f58965 in term 1.
I20260811 00:19:53.344636 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.333687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:33237 (local address 127.0.0.1:46620)
0811 00:19:53.334168 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.334188 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:53.334296 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:53.336241 (+ 1945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:53.336246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:53.336278 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:53.336705 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.336852 (+ 147us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.339657 (+ 2805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.339662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.342138 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.342155 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.343751 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.343758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.343976 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.344024 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 00:19:53.344120 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:19:53.344279 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":44,"thread_start_us":1519,"threads_started":1}
I20260811 00:19:53.344781 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.334785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.16.71.188:33237)
0811 00:19:53.335882 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.335888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.335928 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:19:53.335993 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.335997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.336070 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.337041 (+ 971us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:53.337816 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.339511 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.342355 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.343545 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.343656 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.343744 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.344204 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.344356 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":955,"thread_start_us":819,"threads_started":1}
I20260811 00:19:53.344923 28744 leader_election.cc:302] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c67b5d559af48b7bcf14cdfc1f58965, fbc8732d8d19493c862e2c1519fae618; no voters: 2dada80505ec4b0f87382962092f83b7
I20260811 00:19:53.345534 29704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 00:19:53.345711 29704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2dada80505ec4b0f87382962092f83b7)
I20260811 00:19:53.345666 28792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fbc8732d8d19493c862e2c1519fae618" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" is_pre_election: true
I20260811 00:19:53.346174 29704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:19:53.348522 29704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } }
I20260811 00:19:53.349072 28792 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbc8732d8d19493c862e2c1519fae618 in current term 2: Already voted for candidate 0c67b5d559af48b7bcf14cdfc1f58965 in this term.
I20260811 00:19:53.349723 29704 leader_election.cc:288] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [CANDIDATE]: Term 2 election: Requested vote from peers 2dada80505ec4b0f87382962092f83b7 (127.16.71.190:42327), fbc8732d8d19493c862e2c1519fae618 (127.16.71.189:38833)
I20260811 00:19:53.349927 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.349442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42846)
0811 00:19:53.349638 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.349743 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 00:19:53.350172 28451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fbc8732d8d19493c862e2c1519fae618"
I20260811 00:19:53.350397 28414 leader_election.cc:302] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbc8732d8d19493c862e2c1519fae618; no voters: 0c67b5d559af48b7bcf14cdfc1f58965, 2dada80505ec4b0f87382962092f83b7
I20260811 00:19:53.350644 28451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:19:53.351788 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.351315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42850)
0811 00:19:53.351496 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.351615 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:19:53.352029 28746 leader_election.cc:334] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2dada80505ec4b0f87382962092f83b7 (127.16.71.190:42327): Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
I20260811 00:19:53.353235 28451 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c67b5d559af48b7bcf14cdfc1f58965 in term 2.
I20260811 00:19:53.353452 29706 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:19:53.353677 28744 leader_election.cc:302] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c67b5d559af48b7bcf14cdfc1f58965, fbc8732d8d19493c862e2c1519fae618; no voters: 2dada80505ec4b0f87382962092f83b7
I20260811 00:19:53.354045 29704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:19:53.354513 29704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [term 2 LEADER]: Becoming Leader. State: Replica: 0c67b5d559af48b7bcf14cdfc1f58965, State: Running, Role: LEADER
I20260811 00:19:53.354916 29704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } }
I20260811 00:19:53.356853 29718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c67b5d559af48b7bcf14cdfc1f58965. Latest consensus state: current_term: 2 leader_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:53.357265 29718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:53.358026 29721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:53.362916 29721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:53.363916 29721 catalog_manager.cc:1259] Loaded cluster ID: 51a418b47455482897f68fdfa0314b89
I20260811 00:19:53.364012 29721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:53.365382 29721 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:53.366276 29721 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Loaded TSK: 0
I20260811 00:19:53.367317 29721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:19:53.406239 28773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:53.414124 28773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:53.419143 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.418557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42862)
0811 00:19:53.418778 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.418941 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 00:19:53.419286 28451 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [term 2 FOLLOWER]: Refusing update from remote peer 0c67b5d559af48b7bcf14cdfc1f58965: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 00:19:53.420414 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.419926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42872)
0811 00:19:53.420124 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.420224 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 00:19:53.420272 29718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 00:19:53.420708 28746 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 -> Peer 2dada80505ec4b0f87382962092f83b7 (127.16.71.190:42327): Couldn't send request to peer 2dada80505ec4b0f87382962092f83b7. Status: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 00:19:53.423820 29706 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c67b5d559af48b7bcf14cdfc1f58965. Latest consensus state: current_term: 2 leader_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:53.424158 29706 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:19:53.425796 29718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:53.426156 29718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:53.426748 29706 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dada80505ec4b0f87382962092f83b7" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42327 } } peers { permanent_uuid: "fbc8732d8d19493c862e2c1519fae618" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 38833 } attrs { promote: false } } peers { permanent_uuid: "0c67b5d559af48b7bcf14cdfc1f58965" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 33237 } attrs { promote: false } } }
I20260811 00:19:53.427081 29706 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc8732d8d19493c862e2c1519fae618 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:19:53.450074 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.441662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:41085 (local address 127.0.0.1:58164)
0811 00:19:53.441861 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.441883 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:53.442002 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:53.443354 (+ 1352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:53.443359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:53.443389 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:53.443637 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.443651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.445161 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.445167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.447359 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.447380 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.449145 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.449152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.449417 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.449461 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:19:53.449550 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:19:53.449711 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":1}
I20260811 00:19:53.450819 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.442040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:45607 (local address 127.0.0.1:39434)
0811 00:19:53.442524 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.442541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:53.442658 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:53.444162 (+ 1504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:53.444167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:53.444197 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:53.444508 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.444521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.446114 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.446119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.448230 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.448248 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.450044 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.450051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.450255 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.450292 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:19:53.450365 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 00:19:53.450527 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351}
I20260811 00:19:53.451792 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.442904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.16.71.129:45607)
0811 00:19:53.443792 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.443798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.443815 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:19:53.443898 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.443904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.443994 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.444687 (+ 693us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:53.444878 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.445915 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.448491 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.449737 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.451083 (+ 1346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.451204 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.451269 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.451438 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":1409,"threads_started":1}
I20260811 00:19:53.452109 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.442090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.16.71.130:41085)
0811 00:19:53.443045 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.443050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.443067 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:19:53.443133 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.443137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.443197 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.443298 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:53.443841 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.444974 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.447634 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.448945 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.451295 (+ 2350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.451411 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.451466 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.451638 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":743,"threads_started":1}
I20260811 00:19:53.453223 29235 tablet_service.cc:1467] Processing CreateTablet for tablet 54e82207c1124341aa0c5d840278be07 (DEFAULT_TABLE table=test-table [id=740ae89a3c9a4f56986c9d84d577c78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:53.453675 28975 tablet_service.cc:1467] Processing CreateTablet for tablet 54e82207c1124341aa0c5d840278be07 (DEFAULT_TABLE table=test-table [id=740ae89a3c9a4f56986c9d84d577c78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:53.453789 29235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54e82207c1124341aa0c5d840278be07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:53.454144 28975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54e82207c1124341aa0c5d840278be07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:53.459559 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.444187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.16.71.131:33589)
0811 00:19:53.452832 (+ 8645us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.452839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.452859 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:19:53.452946 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.452951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.453088 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.453256 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:53.453791 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.455124 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.457535 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.458416 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.458563 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.458682 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.459070 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.459233 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8507,"thread_start_us":13150,"threads_started":1}
I20260811 00:19:53.459630 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.443722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:33589 (local address 127.0.0.1:53704)
0811 00:19:53.450755 (+ 7033us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.450776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:53.451098 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:53.453277 (+ 2179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:53.453282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:53.453311 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:53.453591 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.453609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.455292 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.455299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.457300 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.457319 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.458647 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.458654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.458859 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.458915 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 00:19:53.459018 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 00:19:53.459177 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6923,"thread_start_us":8248,"threads_started":1}
I20260811 00:19:53.461669 29475 tablet_service.cc:1467] Processing CreateTablet for tablet 54e82207c1124341aa0c5d840278be07 (DEFAULT_TABLE table=test-table [id=740ae89a3c9a4f56986c9d84d577c78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:53.462144 29475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54e82207c1124341aa0c5d840278be07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:53.468240 29739 tablet_bootstrap.cc:492] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: Bootstrap starting.
I20260811 00:19:53.469985 29737 tablet_bootstrap.cc:492] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: Bootstrap starting.
I20260811 00:19:53.470639 29738 tablet_bootstrap.cc:492] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: Bootstrap starting.
I20260811 00:19:53.470687 29739 tablet_bootstrap.cc:654] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:53.472208 29737 tablet_bootstrap.cc:654] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:53.472970 29738 tablet_bootstrap.cc:654] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:53.476037 29739 tablet_bootstrap.cc:492] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: No bootstrap required, opened a new log
I20260811 00:19:53.476213 29739 ts_tablet_manager.cc:1397] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 00:19:53.476250 29738 tablet_bootstrap.cc:492] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: No bootstrap required, opened a new log
I20260811 00:19:53.476374 29738 ts_tablet_manager.cc:1397] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 00:19:53.477136 29738 raft_consensus.cc:348] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.477154 29739 raft_consensus.cc:348] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.477419 29738 raft_consensus.cc:374] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:53.477420 29739 raft_consensus.cc:374] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:53.477519 29738 raft_consensus.cc:729] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50104b6eb2364c70a066d1547eeb9591, State: Initialized, Role: FOLLOWER
I20260811 00:19:53.477528 29739 raft_consensus.cc:729] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6fd1186d724dd78559db05d7d42c3a, State: Initialized, Role: FOLLOWER
I20260811 00:19:53.477838 29737 tablet_bootstrap.cc:492] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: No bootstrap required, opened a new log
I20260811 00:19:53.477761 29738 consensus_queue.cc:260] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.477813 29739 consensus_queue.cc:260] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.479321 29738 ts_tablet_manager.cc:1428] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 00:19:53.481581 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.478958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:39777)
0811 00:19:53.481157 (+ 2199us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.481344 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2060,"thread_start_us":1901,"threads_started":1}
I20260811 00:19:53.482362 29739 ts_tablet_manager.cc:1428] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 00:19:53.484735 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.482228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:35077)
0811 00:19:53.484366 (+ 2138us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.484485 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2016,"threads_started":1}
I20260811 00:19:53.487789 29737 ts_tablet_manager.cc:1397] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260811 00:19:53.488914 29737 raft_consensus.cc:348] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.489212 29737 raft_consensus.cc:374] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:53.489333 29737 raft_consensus.cc:729] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c41039d6264036a6970bab6d2280f4, State: Initialized, Role: FOLLOWER
I20260811 00:19:53.489650 29737 consensus_queue.cc:260] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.489796 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.486448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.131:60895)
0811 00:19:53.489245 (+ 2797us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.489430 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2618,"thread_start_us":2438,"threads_started":1}
I20260811 00:19:53.490471 29570 heartbeater.cc:503] Master 127.16.71.188:33237 was elected leader, sending a full tablet report...
I20260811 00:19:53.490757 29090 heartbeater.cc:503] Master 127.16.71.188:33237 was elected leader, sending a full tablet report...
I20260811 00:19:53.493446 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.491508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.131:51509)
0811 00:19:53.493067 (+ 1559us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.493176 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1429}
I20260811 00:19:53.493505 29737 ts_tablet_manager.cc:1428] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 00:19:53.496450 29341 heartbeater.cc:503] Master 127.16.71.188:33237 was elected leader, sending a full tablet report...
I20260811 00:19:53.496806 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.495385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.130:59457)
0811 00:19:53.496299 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.496506 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":574,"threads_started":1}
I20260811 00:19:53.498082 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.497654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.130:52705)
0811 00:19:53.497847 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.497942 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 00:19:53.565521 29745 raft_consensus.cc:482] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:53.565824 29745 raft_consensus.cc:504] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.567493 29745 leader_election.cc:288] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085), 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607)
I20260811 00:19:53.571171 29744 raft_consensus.cc:482] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:53.571439 29744 raft_consensus.cc:504] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.572590 29744 leader_election.cc:288] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085), 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
I20260811 00:19:53.572906 29267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e82207c1124341aa0c5d840278be07" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c41039d6264036a6970bab6d2280f4" is_pre_election: true
I20260811 00:19:53.572978 29507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e82207c1124341aa0c5d840278be07" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6fd1186d724dd78559db05d7d42c3a" is_pre_election: true
I20260811 00:19:53.573226 29507 raft_consensus.cc:2454] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 0.
I20260811 00:19:53.573241 29267 raft_consensus.cc:2454] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 0.
I20260811 00:19:53.573781 28889 leader_election.cc:302] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50104b6eb2364c70a066d1547eeb9591, 9d6fd1186d724dd78559db05d7d42c3a; no voters:
I20260811 00:19:53.574271 29744 raft_consensus.cc:2781] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:53.574406 29744 raft_consensus.cc:482] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:53.574504 29744 raft_consensus.cc:3037] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:53.575742 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.567963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:38063 (local address 127.16.71.129:45607)
0811 00:19:53.568159 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.568187 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.568828 (+ 641us) server_negotiation.cc:407] Connection header received
0811 00:19:53.568985 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.568991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.569082 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.569197 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:53.569893 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.570831 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.573394 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.574671 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.574854 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.574949 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.575306 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.575456 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:19:53.575838 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.567895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:45607 (local address 127.16.71.131:38063)
0811 00:19:53.568677 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.568703 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:53.568839 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:53.569249 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:53.569253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:53.569290 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:53.569639 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.569652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.571034 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.571040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.573185 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.573206 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.574867 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.574874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.575102 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.575143 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:19:53.575239 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:19:53.575392 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":1,"spinlock_wait_cycles":896,"thread_start_us":475,"threads_started":1}
I20260811 00:19:53.576956 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.567945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:40359 (local address 127.16.71.130:41085)
0811 00:19:53.568168 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.568174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.568274 (+ 100us) server_negotiation.cc:407] Connection header received
0811 00:19:53.568558 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.568564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.568694 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.568799 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:53.569744 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.570528 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.574390 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.575424 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.575597 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.575685 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.576531 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.576705 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:19:53.576783 29007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e82207c1124341aa0c5d840278be07" candidate_uuid: "9d6fd1186d724dd78559db05d7d42c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50104b6eb2364c70a066d1547eeb9591" is_pre_election: true
I20260811 00:19:53.577283 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.567853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:41085 (local address 127.16.71.131:40359)
0811 00:19:53.568144 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.568167 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:53.568395 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:53.569209 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:53.569214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:53.569242 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:53.569558 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.569568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.570668 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.570674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.574221 (+ 3547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:53.574237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.576198 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.576205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:53.576343 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.576379 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:19:53.576468 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:19:53.576660 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":1,"spinlock_wait_cycles":6784}
I20260811 00:19:53.577793 29744 raft_consensus.cc:504] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.577873 29267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e82207c1124341aa0c5d840278be07" candidate_uuid: "9d6fd1186d724dd78559db05d7d42c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c41039d6264036a6970bab6d2280f4" is_pre_election: true
I20260811 00:19:53.578094 29267 raft_consensus.cc:2454] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d6fd1186d724dd78559db05d7d42c3a in term 0.
I20260811 00:19:53.578231 29007 raft_consensus.cc:2379] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d6fd1186d724dd78559db05d7d42c3a in current term 1: Already voted for candidate 50104b6eb2364c70a066d1547eeb9591 in this term.
I20260811 00:19:53.578603 29744 leader_election.cc:288] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 election: Requested vote from peers 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085), 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589)
I20260811 00:19:53.578819 29267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e82207c1124341aa0c5d840278be07" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c41039d6264036a6970bab6d2280f4"
I20260811 00:19:53.578990 29267 raft_consensus.cc:3037] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:53.579595 29397 leader_election.cc:302] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c41039d6264036a6970bab6d2280f4, 9d6fd1186d724dd78559db05d7d42c3a; no voters:
I20260811 00:19:53.579720 29507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e82207c1124341aa0c5d840278be07" candidate_uuid: "50104b6eb2364c70a066d1547eeb9591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6fd1186d724dd78559db05d7d42c3a"
I20260811 00:19:53.579941 29507 raft_consensus.cc:3037] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:53.581679 29267 raft_consensus.cc:2454] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 1.
I20260811 00:19:53.582132 28897 leader_election.cc:302] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c41039d6264036a6970bab6d2280f4, 50104b6eb2364c70a066d1547eeb9591; no voters:
I20260811 00:19:53.582221 29507 raft_consensus.cc:2454] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50104b6eb2364c70a066d1547eeb9591 in term 1.
I20260811 00:19:53.582406 29745 raft_consensus.cc:2741] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 00:19:53.582887 29744 raft_consensus.cc:2781] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:53.583498 29744 raft_consensus.cc:686] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 1 LEADER]: Becoming Leader. State: Replica: 50104b6eb2364c70a066d1547eeb9591, State: Running, Role: LEADER
I20260811 00:19:53.583887 29744 consensus_queue.cc:237] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } }
I20260811 00:19:53.584952 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.584191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:43519)
0811 00:19:53.584402 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.584678 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 00:19:53.592183 28773 catalog_manager.cc:5495] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "50104b6eb2364c70a066d1547eeb9591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50104b6eb2364c70a066d1547eeb9591" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:53.632761 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.627986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.16.71.129:45607)
0811 00:19:53.628200 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.628205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.628257 (+ 52us) server_negotiation.cc:407] Connection header received
0811 00:19:53.628438 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.628441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.628515 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.628594 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:53.628946 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.629722 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.631044 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.631524 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.631635 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.632100 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:53.632213 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:53.632278 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.632395 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.632467 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 00:19:53.635973 29507 raft_consensus.cc:1261] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Refusing update from remote peer 50104b6eb2364c70a066d1547eeb9591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:53.636375 29267 raft_consensus.cc:1261] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Refusing update from remote peer 50104b6eb2364c70a066d1547eeb9591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:53.637012 29756 consensus_queue.cc:1035] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d6fd1186d724dd78559db05d7d42c3a" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 33589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:53.637458 29744 consensus_queue.cc:1035] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c41039d6264036a6970bab6d2280f4" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 41085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:53.639991 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.639379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.131:59341)
0811 00:19:53.639701 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.639803 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 00:19:53.640897 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.640214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.130:55875)
0811 00:19:53.640479 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.640679 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 00:19:53.648646 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.648108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.130:60667)
0811 00:19:53.648336 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.648435 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 00:19:53.649844 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.649021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:41071)
0811 00:19:53.649586 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.649683 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443}
I20260811 00:19:53.658221 28773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:53.658660 28773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:53.665659 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.664990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.131:50213)
0811 00:19:53.665335 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.665429 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
I20260811 00:19:53.666121 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.664878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.16.71.129:56251)
0811 00:19:53.665701 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.665847 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":661}
I20260811 00:19:53.669171 28773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965: Sending DeleteTablet for 3 replicas of tablet 54e82207c1124341aa0c5d840278be07
I20260811 00:19:53.670325 29475 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e82207c1124341aa0c5d840278be07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:53 UTC) from {username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:53704
I20260811 00:19:53.670317 29235 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e82207c1124341aa0c5d840278be07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:53 UTC) from {username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:58164
I20260811 00:19:53.671162 29778 tablet_replica.cc:330] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: stopping tablet replica
I20260811 00:19:53.671383 29778 raft_consensus.cc:2229] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:53.671799 29778 raft_consensus.cc:2258] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:53.672461 28975 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e82207c1124341aa0c5d840278be07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:53 UTC) from {username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:39434
I20260811 00:19:53.672411 28973 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e82207c1124341aa0c5d840278be07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:53 UTC) from {username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:39434
I20260811 00:19:53.672605 29474 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e82207c1124341aa0c5d840278be07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:53 UTC) from {username='oryx', principal='oryx/127.16.71.188@KRBTEST.COM'} at 127.0.0.1:53704
I20260811 00:19:53.673066 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 28750
I20260811 00:19:53.673743 29779 tablet_replica.cc:330] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: stopping tablet replica
I20260811 00:19:53.673955 29779 raft_consensus.cc:2229] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:53.674273 29779 raft_consensus.cc:2258] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:53.677175 29781 tablet_replica.cc:330] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: stopping tablet replica
I20260811 00:19:53.677474 29781 raft_consensus.cc:2229] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:53.677693 29779 ts_tablet_manager.cc:1905] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:53.678082 29781 raft_consensus.cc:2258] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:53.679090 29781 ts_tablet_manager.cc:1905] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:53.685237 29779 ts_tablet_manager.cc:1918] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:53.685421 29779 log.cc:1196] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/54e82207c1124341aa0c5d840278be07
I20260811 00:19:53.685734 29778 ts_tablet_manager.cc:1905] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:53.685900 29779 ts_tablet_manager.cc:1939] T 54e82207c1124341aa0c5d840278be07 P 9d6fd1186d724dd78559db05d7d42c3a: Deleting consensus metadata
I20260811 00:19:53.686503 29781 ts_tablet_manager.cc:1918] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:53.686709 29781 log.cc:1196] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/54e82207c1124341aa0c5d840278be07
I20260811 00:19:53.687253 29781 ts_tablet_manager.cc:1939] T 54e82207c1124341aa0c5d840278be07 P 50104b6eb2364c70a066d1547eeb9591: Deleting consensus metadata
I20260811 00:19:53.687423 28743 catalog_manager.cc:4841] TS 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589): tablet 54e82207c1124341aa0c5d840278be07 (table test-table [id=740ae89a3c9a4f56986c9d84d577c78a]) successfully deleted
I20260811 00:19:53.688481 28743 catalog_manager.cc:4841] TS 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607): tablet 54e82207c1124341aa0c5d840278be07 (table test-table [id=740ae89a3c9a4f56986c9d84d577c78a]) successfully deleted
W20260811 00:19:53.689051 28743 catalog_manager.cc:4808] TS 9d6fd1186d724dd78559db05d7d42c3a (127.16.71.131:33589): delete failed for tablet 54e82207c1124341aa0c5d840278be07 because the tablet was not found. No further retry: Not found: Tablet not found: 54e82207c1124341aa0c5d840278be07
W20260811 00:19:53.689312 28743 catalog_manager.cc:4808] TS 50104b6eb2364c70a066d1547eeb9591 (127.16.71.129:45607): delete failed for tablet 54e82207c1124341aa0c5d840278be07 because the tablet was not found. No further retry: Not found: Tablet not found: 54e82207c1124341aa0c5d840278be07
I20260811 00:19:53.692811 29778 ts_tablet_manager.cc:1918] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:19:53.693053 29778 log.cc:1196] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/54e82207c1124341aa0c5d840278be07
I20260811 00:19:53.693161 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.674719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.16.71.129:45607)
0811 00:19:53.674931 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.674939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.674980 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:19:53.675203 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.675209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.675306 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.675410 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:53.676892 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.677720 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.679785 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.680284 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.680496 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.683241 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:53.683259 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:53.683277 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:53.683343 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:53.688229 (+ 4886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:53.690302 (+ 2073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:53.690314 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:53.690317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:53.690518 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:53.691031 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:53.691037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:53.691040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:53.691583 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:53.691935 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.692261 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.692755 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:19:53.693572 29778 ts_tablet_manager.cc:1939] T 54e82207c1124341aa0c5d840278be07 P 23c41039d6264036a6970bab6d2280f4: Deleting consensus metadata
I20260811 00:19:53.693988 29080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:53.694834 28747 catalog_manager.cc:4841] TS 23c41039d6264036a6970bab6d2280f4 (127.16.71.130:41085): tablet 54e82207c1124341aa0c5d840278be07 (table test-table [id=740ae89a3c9a4f56986c9d84d577c78a]) successfully deleted
I20260811 00:19:53.820111 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 28750
I20260811 00:19:53.854310 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 29121
I20260811 00:19:53.869695 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.855182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.16.71.130:41085)
0811 00:19:53.855391 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:53.855402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:53.855424 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:19:53.855541 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:53.855545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:53.855621 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:53.855716 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:53.857192 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.858014 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.859428 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:53.859879 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:53.860041 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:53.863038 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:53.863066 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:53.863095 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:53.863167 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:53.866476 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:53.867378 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:53.867388 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:53.867391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:53.867563 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:53.868005 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:53.868011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:53.868013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:53.868472 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:53.868790 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:53.869207 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 00:19:53.869414 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:19:53.870434 29332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:53.969681 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 29121
I20260811 00:19:53.977370 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.975423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42884)
0811 00:19:53.977096 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.977190 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1560,"thread_start_us":1452,"threads_started":1}
I20260811 00:19:53.980280 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:53.979846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42327 (local address 127.0.0.1:42898)
0811 00:19:53.980052 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:53.980141 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:42327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 00:19:53.999660 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 29360
I20260811 00:19:54.014973 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:54.000502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.16.71.131:33589)
0811 00:19:54.001027 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:54.001032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:54.001050 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:19:54.001106 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:54.001109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:54.001172 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:54.001277 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:54.002753 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:54.003546 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:54.004917 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:54.005387 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:54.005506 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:54.008302 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:54.008331 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:54.008348 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:54.008393 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:54.011426 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:54.012345 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:54.012354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:54.012356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:54.012538 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:54.013160 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:54.013164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:54.013166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:54.013605 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:54.013870 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:54.014246 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 00:19:54.014649 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":278,"threads_started":1}
I20260811 00:19:54.015750 29565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:54.112808 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 29360
I20260811 00:19:54.140923 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 28189
I20260811 00:19:54.142284 28494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:19:54.263561 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 28189
W20260811 00:19:54.296120 28744 connection.cc:535] client connection to 127.16.71.189:38833 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:19:54.296413 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 28374
I20260811 00:19:54.297621 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:54.296961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:38833 (local address 127.0.0.1:50566)
0811 00:19:54.297297 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:54.297394 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:38833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 00:19:54.297752 28825 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 00:19:54.297851 28744 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0c67b5d559af48b7bcf14cdfc1f58965 -> Peer fbc8732d8d19493c862e2c1519fae618 (127.16.71.189:38833): Couldn't send request to peer fbc8732d8d19493c862e2c1519fae618. Status: Network error: Client connection negotiation failed: client connection to 127.16.71.189:38833: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 00:19:54.420475 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 28374
2026-08-11T00:19:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10601 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:19:54 dist-test-slave-f7th krb5kdc[29802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:19:54 dist-test-slave-f7th krb5kdc[29802](info): set up 2 sockets
Aug 11 00:19:54 dist-test-slave-f7th krb5kdc[29802](info): commencing operation
krb5kdc: starting...
W20260811 00:19:55.718497 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:19:55 dist-test-slave-f7th krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:19:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:19:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab.
Entry for principal oryx/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab.
I20260811 00:19:56.001051 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34609
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab
--principal=oryx/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:34609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:56.359570 29827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:56.359915 29827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:56.360056 29827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:56.367738 29827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:19:56.367882 29827 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:19:56.367942 29827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:56.368028 29827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:56.368099 29827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:19:56.368191 29827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:19:56.378841 29827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:34609
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.190.keytab
--principal=oryx/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:56.380264 29827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:56.381713 29827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:56.394323 29838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:56.394522 29845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:56.394883 29833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:56.395272 29827 server_base.cc:1034] running on GCE node
I20260811 00:19:56.396180 29827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:56.397847 29827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:56.399137 29827 hybrid_clock.cc:648] HybridClock initialized: now 1786407596399057 us; error 67 us; skew 500 ppm
I20260811 00:19:56.399511 29827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:56 dist-test-slave-f7th krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407596, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:56.402920 29827 init.cc:377] Logged in from keytab as oryx/127.16.71.190@KRBTEST.COM (short username oryx)
I20260811 00:19:56.404721 29827 webserver.cc:460] Webserver started at http://127.16.71.190:38395/ using document root <none> and password file <none>
I20260811 00:19:56.405537 29827 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:56.405659 29827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:56.406003 29827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:56.409446 29827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "f53cfdfc642043dfa8aa4c39e6174312"
format_stamp: "Formatted at 2026-08-11 00:19:56 on dist-test-slave-f7th"
server_key: "e741019fcbfe9caa0f37f3399ad7a5f0"
server_key_iv: "a1209dc88042e074c3380d0bdd8e7cc8"
server_key_version: "encryptionkey@0"
I20260811 00:19:56.410452 29827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "f53cfdfc642043dfa8aa4c39e6174312"
format_stamp: "Formatted at 2026-08-11 00:19:56 on dist-test-slave-f7th"
server_key: "e741019fcbfe9caa0f37f3399ad7a5f0"
server_key_iv: "a1209dc88042e074c3380d0bdd8e7cc8"
server_key_version: "encryptionkey@0"
I20260811 00:19:56.416695 29827 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:19:56.420996 29852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:56.422654 29827 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:19:56.422865 29827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "f53cfdfc642043dfa8aa4c39e6174312"
format_stamp: "Formatted at 2026-08-11 00:19:56 on dist-test-slave-f7th"
server_key: "e741019fcbfe9caa0f37f3399ad7a5f0"
server_key_iv: "a1209dc88042e074c3380d0bdd8e7cc8"
server_key_version: "encryptionkey@0"
I20260811 00:19:56.423153 29827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:56.454631 29827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:56.459450 29827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:56.459740 29827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:56.482215 29827 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:34609
I20260811 00:19:56.482256 29960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:34609 every 8 connection(s)
I20260811 00:19:56.483757 29827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:19:56.488205 29961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:56.490506 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 29827
I20260811 00:19:56.490756 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:19:56.491623 16670 external_mini_cluster.cc:1428] Setting key cd6b2bb5e1d4b680251dd913b0fd8fda
I20260811 00:19:56.503239 29961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Bootstrap starting.
I20260811 00:19:56.507336 29961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:56.508478 29961 log.cc:824] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:19:56 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:56.512406 29961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: No bootstrap required, opened a new log
I20260811 00:19:56.520565 29967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:56.495069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.16.71.190:34609)
0811 00:19:56.496973 (+ 1904us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:56.496987 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:56.497037 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:19:56.498839 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:56.498864 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:56.499307 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:56.499691 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:56.501828 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:56.503306 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:56.504749 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:56.505942 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:56.506153 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:56.510721 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:56.510734 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:56.510755 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:56.510804 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:56.515029 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:56.515928 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:56.515939 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:56.515954 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:56.516214 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:56.516599 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:56.516604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:56.516608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:56.517489 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:56.517829 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:56.518373 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 00:19:56.518959 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1606,"thread_start_us":1281,"threads_started":1}
I20260811 00:19:56.522921 29961 raft_consensus.cc:348] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53cfdfc642043dfa8aa4c39e6174312" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34609 } }
I20260811 00:19:56.523341 29961 raft_consensus.cc:374] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:56.523461 29961 raft_consensus.cc:729] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f53cfdfc642043dfa8aa4c39e6174312, State: Initialized, Role: FOLLOWER
I20260811 00:19:56.524140 29961 consensus_queue.cc:260] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53cfdfc642043dfa8aa4c39e6174312" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34609 } }
I20260811 00:19:56.524379 29961 raft_consensus.cc:388] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:19:56.524488 29961 raft_consensus.cc:482] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:19:56.524691 29961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:56.527268 29961 raft_consensus.cc:504] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53cfdfc642043dfa8aa4c39e6174312" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34609 } }
I20260811 00:19:56.527872 29961 leader_election.cc:302] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f53cfdfc642043dfa8aa4c39e6174312; no voters:
I20260811 00:19:56.528501 29961 leader_election.cc:288] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:19:56.528664 29971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:56.529367 29971 raft_consensus.cc:686] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [term 1 LEADER]: Becoming Leader. State: Replica: f53cfdfc642043dfa8aa4c39e6174312, State: Running, Role: LEADER
I20260811 00:19:56.529951 29971 consensus_queue.cc:237] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53cfdfc642043dfa8aa4c39e6174312" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34609 } }
I20260811 00:19:56.530706 29961 sys_catalog.cc:564] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:19:56.532822 29972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f53cfdfc642043dfa8aa4c39e6174312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53cfdfc642043dfa8aa4c39e6174312" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34609 } } }
I20260811 00:19:56.533257 29972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:56.535163 29980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:19:56.534842 29974 sys_catalog.cc:455] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f53cfdfc642043dfa8aa4c39e6174312. Latest consensus state: current_term: 1 leader_uuid: "f53cfdfc642043dfa8aa4c39e6174312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53cfdfc642043dfa8aa4c39e6174312" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34609 } } }
I20260811 00:19:56.535738 29974 sys_catalog.cc:458] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312 [sys.catalog]: This master's current role is: LEADER
I20260811 00:19:56.540829 29980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:19:56.551496 29980 catalog_manager.cc:1347] Generated new cluster ID: 85677ab5a6854271a592554c245b1714
I20260811 00:19:56.551631 29980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:19:56.578363 29980 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:19:56.580035 29980 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:19:56.589630 29980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Generated new TSK 0
I20260811 00:19:56.590559 29980 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab.
Entry for principal oryx/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab.
I20260811 00:19:56.657446 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34609
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab
--principal=oryx/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:57.019157 30006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:57.019541 30006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:57.019685 30006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:57.027460 30006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:57.027617 30006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:57.027755 30006 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:19:57.027868 30006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:19:57.038615 30006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab
--principal=oryx/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:34609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:57.039996 30006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:57.041436 30006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:57.055814 30014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:57.055845 30017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:57.055874 30019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:57.056641 30006 server_base.cc:1034] running on GCE node
I20260811 00:19:57.057387 30006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:57.058398 30006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:57.059674 30006 hybrid_clock.cc:648] HybridClock initialized: now 1786407597059578 us; error 56 us; skew 500 ppm
I20260811 00:19:57.060065 30006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:57 dist-test-slave-f7th krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407597, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:57.063611 30006 init.cc:377] Logged in from keytab as oryx/127.16.71.129@KRBTEST.COM (short username oryx)
I20260811 00:19:57.065456 30006 webserver.cc:460] Webserver started at http://127.16.71.129:36437/ using document root <none> and password file <none>
I20260811 00:19:57.066246 30006 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:57.066375 30006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:57.066721 30006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:57.070117 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "d07358696fb3463a9e0e844e6b338460"
format_stamp: "Formatted at 2026-08-11 00:19:57 on dist-test-slave-f7th"
server_key: "c73034a8085a208fc34349e535daddfd"
server_key_iv: "5d6fc3b58c6ad70ad930952a007e9d70"
server_key_version: "encryptionkey@0"
I20260811 00:19:57.071041 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "d07358696fb3463a9e0e844e6b338460"
format_stamp: "Formatted at 2026-08-11 00:19:57 on dist-test-slave-f7th"
server_key: "c73034a8085a208fc34349e535daddfd"
server_key_iv: "5d6fc3b58c6ad70ad930952a007e9d70"
server_key_version: "encryptionkey@0"
I20260811 00:19:57.077036 30006 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:19:57.081168 30039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:57.082767 30006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:19:57.083000 30006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "d07358696fb3463a9e0e844e6b338460"
format_stamp: "Formatted at 2026-08-11 00:19:57 on dist-test-slave-f7th"
server_key: "c73034a8085a208fc34349e535daddfd"
server_key_iv: "5d6fc3b58c6ad70ad930952a007e9d70"
server_key_version: "encryptionkey@0"
I20260811 00:19:57.083254 30006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:57.115087 30006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:57.119901 30006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:57.120206 30006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:57.123266 30006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:57.123363 30006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:57.123517 30006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:57.123596 30006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:57 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407597, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:57.150615 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.124310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:49401 (local address 127.16.71.190:34609)
0811 00:19:57.124796 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:57.124802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:57.126882 (+ 2080us) server_negotiation.cc:407] Connection header received
0811 00:19:57.130183 (+ 3301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:57.130191 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:57.130304 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:57.130426 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:57.133127 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.134263 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.137081 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.137525 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.137667 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.141864 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:57.141873 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:57.141878 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:57.141930 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:57.145505 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.146768 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.146775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.146779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.146948 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.147598 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.147603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.147607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.147990 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:57.148162 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:57.149851 (+ 1689us) server_negotiation.cc:299] Negotiation successful
0811 00:19:57.150162 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
I20260811 00:19:57.151417 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.125543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34609 (local address 127.16.71.129:49401)
0811 00:19:57.126539 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:57.126615 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:57.129669 (+ 3054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:57.130820 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:57.130833 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:57.131786 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:57.132862 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.132900 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.134511 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.134521 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.136852 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.136873 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.137796 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.137805 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.138101 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.138820 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:57.138846 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:57.141486 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:57.145786 (+ 4300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.145798 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.145817 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.146603 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.147073 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.147079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.147083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.147407 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.148251 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:57.148260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:57.148955 (+ 695us) client_negotiation.cc:769] Sending connection context
0811 00:19:57.149369 (+ 414us) client_negotiation.cc:240] Negotiation successful
0811 00:19:57.150029 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":375,"threads_started":1}
I20260811 00:19:57.174109 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.158005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34609 (local address 127.16.71.129:44701)
0811 00:19:57.158738 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:57.158763 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:57.158947 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:57.159295 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:57.159300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:57.160018 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:57.160473 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.160491 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.161714 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.161720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.163530 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.163557 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.164568 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.164577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.164821 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.165693 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:57.165714 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:57.166893 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:57.171021 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.171027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.171029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.171472 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.171898 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.171902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.171905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.172037 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.172645 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:57.172651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:57.172863 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 00:19:57.173036 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 00:19:57.173578 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":346,"threads_started":1}
I20260811 00:19:57.174204 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.158076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:44701 (local address 127.16.71.190:34609)
0811 00:19:57.158317 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:57.158325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:57.158868 (+ 543us) server_negotiation.cc:407] Connection header received
0811 00:19:57.159073 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:57.159077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:57.159150 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:57.159239 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:57.160778 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.161547 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.163740 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.164294 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.164456 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.167302 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:57.167310 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:57.167323 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:57.167374 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:57.170854 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.171622 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.171629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.171633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.171778 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.172159 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.172165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.172169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.172480 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:57.172668 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:57.173593 (+ 925us) server_negotiation.cc:299] Negotiation successful
0811 00:19:57.173857 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:19:57.181710 30006 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:46207
I20260811 00:19:57.181748 30251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:46207 every 8 connection(s)
I20260811 00:19:57.182993 30006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:19:57.185377 30252 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34609
I20260811 00:19:57.185653 30252 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:57.186455 30252 heartbeater.cc:511] Master 127.16.71.190:34609 requested a full tablet report, sending...
I20260811 00:19:57.186879 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 30006
I20260811 00:19:57.187108 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:19:57.187934 16670 external_mini_cluster.cc:1428] Setting key ed1a1e8222700aa5e96963cf1ff0f7d7
I20260811 00:19:57.189162 29881 ts_manager.cc:194] Registered new tserver with Master: d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207)
I20260811 00:19:57.190912 29881 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:49401
WARNING: no policy specified for oryx/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab.
Entry for principal oryx/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab.
I20260811 00:19:57.249910 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34609
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab
--principal=oryx/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:57.606415 30258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:57.606765 30258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:57.606928 30258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:57.614706 30258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:57.614867 30258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:57.614982 30258 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:19:57.615100 30258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:19:57.625967 30258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab
--principal=oryx/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:34609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:57.627619 30258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:57.629274 30258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:57.644323 30279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:57.644389 30285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:57.644407 30281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:57.645310 30258 server_base.cc:1034] running on GCE node
I20260811 00:19:57.646026 30258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:57.647186 30258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:57.648473 30258 hybrid_clock.cc:648] HybridClock initialized: now 1786407597648377 us; error 63 us; skew 500 ppm
I20260811 00:19:57.648849 30258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:57 dist-test-slave-f7th krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407597, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:57.652519 30258 init.cc:377] Logged in from keytab as oryx/127.16.71.130@KRBTEST.COM (short username oryx)
I20260811 00:19:57.654327 30258 webserver.cc:460] Webserver started at http://127.16.71.130:41309/ using document root <none> and password file <none>
I20260811 00:19:57.655107 30258 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:57.655227 30258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:57.655668 30258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:57.658908 30258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "8cccde774423488695a74daefce09560"
format_stamp: "Formatted at 2026-08-11 00:19:57 on dist-test-slave-f7th"
server_key: "a2b979bb040310532806f764c0baa648"
server_key_iv: "c82544886a6faf60e15b6ce1956526c2"
server_key_version: "encryptionkey@0"
I20260811 00:19:57.659818 30258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "8cccde774423488695a74daefce09560"
format_stamp: "Formatted at 2026-08-11 00:19:57 on dist-test-slave-f7th"
server_key: "a2b979bb040310532806f764c0baa648"
server_key_iv: "c82544886a6faf60e15b6ce1956526c2"
server_key_version: "encryptionkey@0"
I20260811 00:19:57.665596 30258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 00:19:57.669724 30293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:57.671319 30258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:19:57.671521 30258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "8cccde774423488695a74daefce09560"
format_stamp: "Formatted at 2026-08-11 00:19:57 on dist-test-slave-f7th"
server_key: "a2b979bb040310532806f764c0baa648"
server_key_iv: "c82544886a6faf60e15b6ce1956526c2"
server_key_version: "encryptionkey@0"
I20260811 00:19:57.671766 30258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:57.691584 30258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:57.696116 30258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:57.696405 30258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:57.699340 30258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:57.699450 30258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:57.699625 30258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:57.699688 30258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:57 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407597, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:57.726111 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.700213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:36989 (local address 127.16.71.190:34609)
0811 00:19:57.700777 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:57.700783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:57.702213 (+ 1430us) server_negotiation.cc:407] Connection header received
0811 00:19:57.704371 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:57.704377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:57.704484 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:57.704582 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:57.707191 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.708300 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.711472 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.712055 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.712221 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.718154 (+ 5933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:57.718160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:57.718163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:57.718203 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:57.721750 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.722981 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.722989 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.722993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.723205 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.723916 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.723921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.723923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.724232 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:57.724403 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:57.725390 (+ 987us) server_negotiation.cc:299] Negotiation successful
0811 00:19:57.725692 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":256,"threads_started":1}
I20260811 00:19:57.727725 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.701004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34609 (local address 127.16.71.130:36989)
0811 00:19:57.701992 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:57.702048 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:57.704026 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:57.704923 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:57.704937 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:57.705845 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:57.706839 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.706878 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.708538 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.708544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.711162 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.711192 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.712665 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.712681 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.713025 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.713859 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:57.713902 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:57.717829 (+ 3927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:57.721962 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.721975 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.721993 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.722772 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.723358 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.723363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.723367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.723758 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.724382 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:57.724392 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:57.724924 (+ 532us) client_negotiation.cc:769] Sending connection context
0811 00:19:57.725233 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 00:19:57.725935 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":415,"threads_started":1}
I20260811 00:19:57.746205 30258 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:34911
I20260811 00:19:57.746214 30524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:34911 every 8 connection(s)
I20260811 00:19:57.748159 30258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:19:57.749814 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 30258
I20260811 00:19:57.750047 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:19:57.750634 16670 external_mini_cluster.cc:1428] Setting key 889353912e293a79022cdd4eea908c62
I20260811 00:19:57.751065 30525 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34609
I20260811 00:19:57.751741 30525 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:57.752571 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.735616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:41431 (local address 127.16.71.190:34609)
0811 00:19:57.735848 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:57.735855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:57.736515 (+ 660us) server_negotiation.cc:407] Connection header received
0811 00:19:57.736874 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:57.736880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:57.736992 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:57.737145 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:57.738763 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.739761 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.742139 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.742746 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.742909 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.745672 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:57.745678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:57.745681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:57.745720 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:57.749079 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.750178 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.750185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.750189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.750378 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:57.750838 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:57.750843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:57.750847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:57.751277 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:57.751442 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:57.751963 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 00:19:57.752237 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:19:57.752717 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:57.735615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34609 (local address 127.16.71.130:41431)
0811 00:19:57.736364 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:57.736386 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:57.736594 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:57.737171 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:57.737175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:57.737906 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:57.738501 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.738535 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.739950 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.739957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.741902 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:57.741926 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:57.743255 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:57.743264 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:57.743498 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:57.744308 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:57.744330 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:57.745404 (+ 1074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:57.749331 (+ 3927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.749338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.749342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.749977 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.750534 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:57.750539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:57.750544 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:57.750723 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:57.751429 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:57.751434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:57.751654 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 00:19:57.751804 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 00:19:57.752183 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":381,"threads_started":1}
I20260811 00:19:57.752827 30525 heartbeater.cc:511] Master 127.16.71.190:34609 requested a full tablet report, sending...
I20260811 00:19:57.754344 29881 ts_manager.cc:194] Registered new tserver with Master: 8cccde774423488695a74daefce09560 (127.16.71.130:34911)
I20260811 00:19:57.755450 29881 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:36989
WARNING: no policy specified for oryx/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab.
Entry for principal oryx/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab.
I20260811 00:19:57.816934 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34609
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab
--principal=oryx/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:19:58.183218 30533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:19:58.183530 30533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:19:58.183655 30533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:19:58.191810 30533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:19:58.191963 30533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:19:58.192080 30533 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:19:58.192185 30533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:19:58.194478 30252 heartbeater.cc:503] Master 127.16.71.190:34609 was elected leader, sending a full tablet report...
I20260811 00:19:58.203116 30533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab
--principal=oryx/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.71.190:34609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:19:58.204721 30533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:19:58.206256 30533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:19:58.221470 30538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:58.221495 30543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:19:58.221441 30547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:19:58.222394 30533 server_base.cc:1034] running on GCE node
I20260811 00:19:58.223146 30533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:19:58.224267 30533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:19:58.225539 30533 hybrid_clock.cc:648] HybridClock initialized: now 1786407598225405 us; error 105 us; skew 500 ppm
I20260811 00:19:58.225909 30533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:19:58 dist-test-slave-f7th krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407598, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:19:58.230057 30533 init.cc:377] Logged in from keytab as oryx/127.16.71.131@KRBTEST.COM (short username oryx)
I20260811 00:19:58.232048 30533 webserver.cc:460] Webserver started at http://127.16.71.131:42699/ using document root <none> and password file <none>
I20260811 00:19:58.232954 30533 fs_manager.cc:359] Metadata directory not provided
I20260811 00:19:58.233074 30533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:19:58.233385 30533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:19:58.236850 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "abb474f603a9490e82a2f69e9f2e3403"
format_stamp: "Formatted at 2026-08-11 00:19:58 on dist-test-slave-f7th"
server_key: "0bb8eb11750b9abaeefc6f02ac918eff"
server_key_iv: "9db05238bb7aad4cd3eb066f38748ca1"
server_key_version: "encryptionkey@0"
I20260811 00:19:58.237831 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "abb474f603a9490e82a2f69e9f2e3403"
format_stamp: "Formatted at 2026-08-11 00:19:58 on dist-test-slave-f7th"
server_key: "0bb8eb11750b9abaeefc6f02ac918eff"
server_key_iv: "9db05238bb7aad4cd3eb066f38748ca1"
server_key_version: "encryptionkey@0"
I20260811 00:19:58.243825 30533 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:19:58.248208 30555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:58.249998 30533 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:19:58.250214 30533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "abb474f603a9490e82a2f69e9f2e3403"
format_stamp: "Formatted at 2026-08-11 00:19:58 on dist-test-slave-f7th"
server_key: "0bb8eb11750b9abaeefc6f02ac918eff"
server_key_iv: "9db05238bb7aad4cd3eb066f38748ca1"
server_key_version: "encryptionkey@0"
I20260811 00:19:58.250475 30533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:19:58.266659 30533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:19:58.271489 30533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:19:58.271759 30533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:19:58.274926 30533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:19:58.275056 30533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:19:58.275311 30533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:19:58.275455 30533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:19:58 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407598, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:58.301501 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:58.275724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:50739 (local address 127.16.71.190:34609)
0811 00:19:58.276565 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:58.276570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:58.277739 (+ 1169us) server_negotiation.cc:407] Connection header received
0811 00:19:58.279992 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:58.279999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:58.280102 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:58.280207 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:58.282879 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.283961 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.286775 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.287403 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.287578 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:58.293009 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:58.293017 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:58.293023 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:58.293073 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:58.297216 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:58.298457 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:58.298462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:58.298465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:58.298628 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:58.299377 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:58.299384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:58.299389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:58.299829 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:58.300007 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:58.300790 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 00:19:58.301056 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":569,"threads_started":1}
I20260811 00:19:58.302650 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:58.276475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34609 (local address 127.16.71.131:50739)
0811 00:19:58.277551 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:58.277602 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:58.279639 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:58.280575 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:58.280593 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:58.281548 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:58.282553 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:58.282593 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.284183 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.284190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:58.286569 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:58.286587 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.287727 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.287736 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:58.288087 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:58.289041 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:58.289078 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:58.292567 (+ 3489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:58.297460 (+ 4893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:58.297479 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:58.297503 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:58.298289 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:58.298795 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:58.298804 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:58.298808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:58.299188 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:58.299983 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:58.299990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:58.300384 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 00:19:58.300669 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 00:19:58.301314 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":451,"threads_started":1}
I20260811 00:19:58.330031 30533 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:41065
I20260811 00:19:58.330107 30762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:41065 every 8 connection(s)
I20260811 00:19:58.331552 30533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:19:58.334107 30763 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34609
I20260811 00:19:58.334388 30763 heartbeater.cc:464] Registering TS with master...
I20260811 00:19:58.335355 30763 heartbeater.cc:511] Master 127.16.71.190:34609 requested a full tablet report, sending...
I20260811 00:19:58.336731 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:58.319462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:51565 (local address 127.16.71.190:34609)
0811 00:19:58.319679 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:58.319685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:58.320475 (+ 790us) server_negotiation.cc:407] Connection header received
0811 00:19:58.320674 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:58.320681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:58.320796 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:58.320940 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:58.322157 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.323365 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.326050 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.326713 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.326893 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:58.329758 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:58.329789 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:58.329794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:58.329843 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:58.333234 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:58.334493 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:58.334500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:58.334504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:58.334698 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:58.335125 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:58.335131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:58.335134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:58.335535 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:58.335738 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:58.336102 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 00:19:58.336383 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:19:58.336762 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:58.319595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34609 (local address 127.16.71.131:51565)
0811 00:19:58.320329 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:58.320348 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:58.320513 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:58.320995 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:58.321000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:58.321601 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:19:58.321952 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:58.321965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.323571 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.323582 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:58.325789 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:58.325823 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.326982 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.326991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:58.327208 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:58.328152 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:19:58.328176 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:19:58.329485 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:19:58.333414 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:58.333420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:58.333424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:58.334306 (+ 882us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:58.334831 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:19:58.334836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:19:58.334840 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:19:58.335001 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:19:58.335657 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:19:58.335661 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:19:58.335834 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 00:19:58.335969 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 00:19:58.336258 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":419,"threads_started":1}
I20260811 00:19:58.337237 29881 ts_manager.cc:194] Registered new tserver with Master: abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:19:58.337291 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 30533
I20260811 00:19:58.337483 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:19:58.338107 16670 external_mini_cluster.cc:1428] Setting key 2192c13b5f21b090c4d6452886bba4d5
I20260811 00:19:58.338330 29881 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:50739
I20260811 00:19:58.342495 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:19:58 dist-test-slave-f7th krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:19:58 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:19:58 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:19:58.758729 30525 heartbeater.cc:503] Master 127.16.71.190:34609 was elected leader, sending a full tablet report...
I20260811 00:19:59.341609 30763 heartbeater.cc:503] Master 127.16.71.190:34609 was elected leader, sending a full tablet report...
W20260811 00:19:59.408592 30773 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.008s user 0.007s sys 0.011s
W20260811 00:19:59.409981 30568 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:19:58.394573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.16.71.190:34609)
0811 00:19:58.394820 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:58.394826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:58.395025 (+ 199us) server_negotiation.cc:407] Connection header received
0811 00:19:58.395190 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:58.395193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:58.395262 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:58.395379 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:58.396770 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.397535 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.398915 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:58.399384 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:58.399512 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.409641 (+1010129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60588: BlockingRecv error: recv got EOF from 127.0.0.1:60588 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 11 00:19:59 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:19:59.429943 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.414453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.16.71.190:34609)
0811 00:19:59.414722 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:59.414729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:59.415006 (+ 277us) server_negotiation.cc:407] Connection header received
0811 00:19:59.415201 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:59.415206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:59.415273 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:59.415398 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:19:59.416775 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.417522 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.418904 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.419392 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.419520 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.423893 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:19:59.423899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:19:59.423903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:19:59.423965 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:19:59.426887 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:59.427753 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:59.427758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:59.427761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:59.427914 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:19:59.428311 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:19:59.428315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:19:59.428317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:19:59.428702 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:19:59.428972 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:59.429278 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 00:19:59.429546 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 00:19:59.434827 29881 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:59.442723 29881 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:59.469669 30796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.457143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46207 (local address 127.0.0.1:39146)
0811 00:19:59.458386 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:59.458439 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:59.458633 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:59.459820 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:59.459829 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:59.459861 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:59.460372 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.460389 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.462511 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.462517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.465465 (+ 2948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.465490 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.468301 (+ 2811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.468310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.468608 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.468712 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 00:19:59.468898 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 00:19:59.469169 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":743,"threads_started":1}
I20260811 00:19:59.472908 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.457050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.16.71.129:46207)
0811 00:19:59.459178 (+ 2128us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:59.459188 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:59.459221 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:19:59.459315 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:59.459321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:59.459563 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:59.459746 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:59.460597 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.462301 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.465798 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.467622 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.467807 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.468100 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:59.472373 (+ 4273us) server_negotiation.cc:299] Negotiation successful
0811 00:19:59.472593 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1913,"thread_start_us":1622,"threads_started":1}
I20260811 00:19:59.478147 30123 tablet_service.cc:1467] Processing CreateTablet for tablet ce95b4dbabe84df982421ad8afcbf6ff (DEFAULT_TABLE table=test-table [id=ad74f6d07efe4664ae590df23cb7b73e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:59.479544 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.457685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.16.71.130:34911)
0811 00:19:59.459199 (+ 1514us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:59.459206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:59.470646 (+ 11440us) server_negotiation.cc:407] Connection header received
0811 00:19:59.470782 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:59.470791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:59.471147 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:59.471379 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:59.472195 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.473736 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.476243 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.478251 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.478427 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.478698 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:59.478981 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 00:19:59.479200 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1364,"thread_start_us":1253,"threads_started":1}
I20260811 00:19:59.479657 30796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.457160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34911 (local address 127.0.0.1:35912)
0811 00:19:59.470296 (+ 13136us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:59.470323 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:59.470469 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:59.471640 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:59.471645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:59.471673 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:59.471965 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.471982 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.473920 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.473924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.475851 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.475880 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.478492 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.478500 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.478715 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.478755 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:19:59.478885 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 00:19:59.479069 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13008,"spinlock_wait_cycles":13696,"thread_start_us":14784,"threads_started":1}
I20260811 00:19:59.480589 30798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.461054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:41065 (local address 127.0.0.1:58162)
0811 00:19:59.471015 (+ 9961us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:59.471035 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:59.471149 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:59.473333 (+ 2184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:59.473339 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:59.473373 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:59.473766 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.473780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.476047 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.476054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.477703 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.477721 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.479766 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.479788 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.480008 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.480038 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:19:59.480108 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 00:19:59.480255 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9834,"thread_start_us":10174,"threads_started":1}
I20260811 00:19:59.480944 30123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce95b4dbabe84df982421ad8afcbf6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:59.481501 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.461712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58162 (local address 127.16.71.131:41065)
0811 00:19:59.472565 (+ 10853us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:59.472577 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:59.472610 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:19:59.472748 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:59.472757 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:59.473043 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:59.473322 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:59.474032 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.475892 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.477925 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.479354 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.480268 (+ 914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.480550 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:59.480998 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 00:19:59.481208 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10658,"thread_start_us":10510,"threads_started":1}
I20260811 00:19:59.484522 30394 tablet_service.cc:1467] Processing CreateTablet for tablet ce95b4dbabe84df982421ad8afcbf6ff (DEFAULT_TABLE table=test-table [id=ad74f6d07efe4664ae590df23cb7b73e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:59.486325 30394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce95b4dbabe84df982421ad8afcbf6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:59.485836 30639 tablet_service.cc:1467] Processing CreateTablet for tablet ce95b4dbabe84df982421ad8afcbf6ff (DEFAULT_TABLE table=test-table [id=ad74f6d07efe4664ae590df23cb7b73e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:59.487623 30639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce95b4dbabe84df982421ad8afcbf6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:59.503495 30803 tablet_bootstrap.cc:492] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Bootstrap starting.
I20260811 00:19:59.507994 30803 tablet_bootstrap.cc:654] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:59.508491 30805 tablet_bootstrap.cc:492] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Bootstrap starting.
I20260811 00:19:59.510432 30804 tablet_bootstrap.cc:492] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Bootstrap starting.
I20260811 00:19:59.514258 30805 tablet_bootstrap.cc:654] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:59.514343 30804 tablet_bootstrap.cc:654] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Neither blocks nor log segments found. Creating new log.
I20260811 00:19:59.515456 30804 log.cc:824] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:59.515446 30805 log.cc:824] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:59.517026 30803 log.cc:824] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Log is configured to *not* fsync() on all Append() calls
I20260811 00:19:59.519102 30804 tablet_bootstrap.cc:492] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: No bootstrap required, opened a new log
I20260811 00:19:59.519172 30805 tablet_bootstrap.cc:492] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: No bootstrap required, opened a new log
I20260811 00:19:59.519387 30804 ts_tablet_manager.cc:1397] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 00:19:59.519493 30805 ts_tablet_manager.cc:1397] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 00:19:59.520475 30803 tablet_bootstrap.cc:492] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: No bootstrap required, opened a new log
I20260811 00:19:59.520816 30803 ts_tablet_manager.cc:1397] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.000s
I20260811 00:19:59.527487 30804 raft_consensus.cc:348] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.527904 30804 raft_consensus.cc:374] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:59.528026 30804 raft_consensus.cc:729] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cccde774423488695a74daefce09560, State: Initialized, Role: FOLLOWER
I20260811 00:19:59.528738 30804 consensus_queue.cc:260] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.530690 30804 ts_tablet_manager.cc:1428] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260811 00:19:59.530372 30805 raft_consensus.cc:348] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.530843 30805 raft_consensus.cc:374] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:59.530956 30805 raft_consensus.cc:729] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb474f603a9490e82a2f69e9f2e3403, State: Initialized, Role: FOLLOWER
I20260811 00:19:59.531643 30805 consensus_queue.cc:260] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.531605 30803 raft_consensus.cc:348] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.532130 30803 raft_consensus.cc:374] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:19:59.532253 30803 raft_consensus.cc:729] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07358696fb3463a9e0e844e6b338460, State: Initialized, Role: FOLLOWER
I20260811 00:19:59.533063 30803 consensus_queue.cc:260] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.533772 30805 ts_tablet_manager.cc:1428] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 00:19:59.535493 30803 ts_tablet_manager.cc:1428] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260811 00:19:59.585364 30764 tablet.cc:2369] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:19:59.686738 30253 tablet.cc:2369] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:19:59.753362 30527 tablet.cc:2369] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:19:59.855816 30821 raft_consensus.cc:482] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:19:59.856166 30821 raft_consensus.cc:504] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.858261 30821 leader_election.cc:288] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:19:59.865983 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.858775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:55209 (local address 127.16.71.129:46207)
0811 00:19:59.859004 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:59.859011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:59.859791 (+ 780us) server_negotiation.cc:407] Connection header received
0811 00:19:59.860082 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:59.860088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:59.860187 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:59.860263 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:59.861173 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.862039 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.864120 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.865017 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.865161 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.865244 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:59.865596 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 00:19:59.865738 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:19:59.866010 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.860041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:39573 (local address 127.16.71.131:41065)
0811 00:19:59.860265 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:59.860271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:59.860400 (+ 129us) server_negotiation.cc:407] Connection header received
0811 00:19:59.860551 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:59.860556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:59.860686 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:59.860814 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:19:59.861530 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.862306 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.864221 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.865095 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.865263 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.865349 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:59.865632 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 00:19:59.865765 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:19:59.866117 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.859674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:41065 (local address 127.16.71.130:39573)
0811 00:19:59.860284 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:59.860305 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:59.860435 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:59.860864 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:59.860868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:59.860896 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:59.861356 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.861369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.862536 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.862540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.864045 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.864063 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.865336 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.865348 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.865470 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.865494 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:19:59.865587 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:19:59.865721 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":297,"threads_started":1}
I20260811 00:19:59.866158 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.858761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46207 (local address 127.16.71.130:55209)
0811 00:19:59.859524 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 00:19:59.859542 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:19:59.859799 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:19:59.860520 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:19:59.860524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:19:59.860553 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:19:59.860989 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.861002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.862195 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.862201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.863918 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:19:59.863933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.865251 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.865257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:19:59.865429 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.865456 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 00:19:59.865534 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 00:19:59.865680 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":442,"threads_started":1}
I20260811 00:19:59.867326 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce95b4dbabe84df982421ad8afcbf6ff" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403" is_pre_election: true
I20260811 00:19:59.867336 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce95b4dbabe84df982421ad8afcbf6ff" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460" is_pre_election: true
I20260811 00:19:59.867976 30679 raft_consensus.cc:2454] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 0.
I20260811 00:19:59.867976 30166 raft_consensus.cc:2454] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 0.
I20260811 00:19:59.868752 30299 leader_election.cc:302] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, d07358696fb3463a9e0e844e6b338460; no voters:
I20260811 00:19:59.869213 30821 raft_consensus.cc:2781] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:19:59.869387 30821 raft_consensus.cc:482] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:19:59.869498 30821 raft_consensus.cc:3037] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:59.872274 30821 raft_consensus.cc:504] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.873064 30821 leader_election.cc:288] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 election: Requested vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:19:59.873802 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce95b4dbabe84df982421ad8afcbf6ff" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460"
I20260811 00:19:59.873826 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce95b4dbabe84df982421ad8afcbf6ff" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403"
I20260811 00:19:59.874082 30679 raft_consensus.cc:3037] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:59.874102 30166 raft_consensus.cc:3037] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:19:59.877576 30679 raft_consensus.cc:2454] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 1.
I20260811 00:19:59.877576 30166 raft_consensus.cc:2454] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 1.
I20260811 00:19:59.878130 30299 leader_election.cc:302] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, abb474f603a9490e82a2f69e9f2e3403; no voters:
I20260811 00:19:59.878476 30821 raft_consensus.cc:2781] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:19:59.878940 30821 raft_consensus.cc:686] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 1 LEADER]: Becoming Leader. State: Replica: 8cccde774423488695a74daefce09560, State: Running, Role: LEADER
I20260811 00:19:59.879457 30821 consensus_queue.cc:237] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:19:59.884526 29878 catalog_manager.cc:5495] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cccde774423488695a74daefce09560 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } health_report { overall_health: UNKNOWN } } }
I20260811 00:19:59.929019 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:19:59.922591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.16.71.130:34911)
0811 00:19:59.922799 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:19:59.922804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:19:59.922847 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:19:59.923138 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:19:59.923142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:19:59.923247 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:19:59.923396 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:19:59.923807 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.924554 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.926081 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:19:59.926569 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:19:59.926683 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:19:59.927418 (+ 735us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:19:59.928304 (+ 886us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:19:59.928437 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:19:59.928661 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 00:19:59.928753 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:19:59.938553 30679 raft_consensus.cc:1261] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Refusing update from remote peer 8cccde774423488695a74daefce09560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:59.938555 30166 raft_consensus.cc:1261] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Refusing update from remote peer 8cccde774423488695a74daefce09560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:19:59.939749 30835 consensus_queue.cc:1035] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:59.940169 30821 consensus_queue.cc:1035] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:19:59.967955 29878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:59.968236 29878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:19:59.972011 29878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Sending DeleteTablet for 3 replicas of tablet ce95b4dbabe84df982421ad8afcbf6ff
I20260811 00:19:59.973554 30394 tablet_service.cc:1514] Processing DeleteTablet for tablet ce95b4dbabe84df982421ad8afcbf6ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:59 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:35912
I20260811 00:19:59.973816 30639 tablet_service.cc:1514] Processing DeleteTablet for tablet ce95b4dbabe84df982421ad8afcbf6ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:59 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:58162
I20260811 00:19:59.974393 30123 tablet_service.cc:1514] Processing DeleteTablet for tablet ce95b4dbabe84df982421ad8afcbf6ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:19:59 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:39146
I20260811 00:19:59.974836 29878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:19:59.975615 29878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:19:59.977525 30863 tablet_replica.cc:330] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: stopping tablet replica
I20260811 00:19:59.978056 30863 raft_consensus.cc:2229] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:59.978631 30863 raft_consensus.cc:2258] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:59.979559 30860 tablet_replica.cc:330] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: stopping tablet replica
I20260811 00:19:59.982728 30860 raft_consensus.cc:2229] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:19:59.985535 30861 tablet_replica.cc:330] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: stopping tablet replica
I20260811 00:19:59.985841 30123 tablet_service.cc:1467] Processing CreateTablet for tablet 11283776d96a4e5590fcf7d138fff942 (DEFAULT_TABLE table=test-table [id=dbd1132b3fb64b42ac2eca427e085efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:59.985848 30394 tablet_service.cc:1467] Processing CreateTablet for tablet 11283776d96a4e5590fcf7d138fff942 (DEFAULT_TABLE table=test-table [id=dbd1132b3fb64b42ac2eca427e085efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:59.986096 30861 raft_consensus.cc:2229] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:19:59.986398 30394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11283776d96a4e5590fcf7d138fff942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:59.986411 30123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11283776d96a4e5590fcf7d138fff942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:59.991372 30860 raft_consensus.cc:2258] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:59.992201 30861 raft_consensus.cc:2258] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:19:59.992961 30863 ts_tablet_manager.cc:1905] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:19:59.992820 30637 tablet_service.cc:1467] Processing CreateTablet for tablet 11283776d96a4e5590fcf7d138fff942 (DEFAULT_TABLE table=test-table [id=dbd1132b3fb64b42ac2eca427e085efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:19:59.993469 30637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11283776d96a4e5590fcf7d138fff942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:19:59.994397 30803 tablet_bootstrap.cc:492] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: Bootstrap starting.
I20260811 00:19:59.996218 30803 tablet_bootstrap.cc:654] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.000255 30863 ts_tablet_manager.cc:1918] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:00.000516 30863 log.cc:1196] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/ce95b4dbabe84df982421ad8afcbf6ff
I20260811 00:20:00.001257 30863 ts_tablet_manager.cc:1939] T ce95b4dbabe84df982421ad8afcbf6ff P d07358696fb3463a9e0e844e6b338460: Deleting consensus metadata
I20260811 00:20:00.001302 30805 tablet_bootstrap.cc:492] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: Bootstrap starting.
I20260811 00:20:00.002696 30804 tablet_bootstrap.cc:492] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: Bootstrap starting.
I20260811 00:20:00.002874 30860 ts_tablet_manager.cc:1905] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:00.003104 30805 tablet_bootstrap.cc:654] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.004473 30804 tablet_bootstrap.cc:654] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.004467 29860 catalog_manager.cc:4841] TS d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207): tablet ce95b4dbabe84df982421ad8afcbf6ff (table test-table [id=ad74f6d07efe4664ae590df23cb7b73e]) successfully deleted
I20260811 00:20:00.005534 30861 ts_tablet_manager.cc:1905] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:00.014219 30804 tablet_bootstrap.cc:492] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: No bootstrap required, opened a new log
I20260811 00:20:00.014520 30804 ts_tablet_manager.cc:1397] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.006s
I20260811 00:20:00.015614 30804 raft_consensus.cc:348] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.015954 30804 raft_consensus.cc:374] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.016146 30804 raft_consensus.cc:729] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cccde774423488695a74daefce09560, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.016538 30804 consensus_queue.cc:260] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.017313 30804 ts_tablet_manager.cc:1428] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 00:20:00.021293 30803 tablet_bootstrap.cc:492] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: No bootstrap required, opened a new log
I20260811 00:20:00.021490 30803 ts_tablet_manager.cc:1397] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.009s
I20260811 00:20:00.022420 30803 raft_consensus.cc:348] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.022729 30803 raft_consensus.cc:374] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.022828 30803 raft_consensus.cc:729] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07358696fb3463a9e0e844e6b338460, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.023161 30803 consensus_queue.cc:260] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.024400 30803 ts_tablet_manager.cc:1428] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:20:00.024832 30861 ts_tablet_manager.cc:1918] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:00.025043 30861 log.cc:1196] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/ce95b4dbabe84df982421ad8afcbf6ff
I20260811 00:20:00.025676 30861 ts_tablet_manager.cc:1939] T ce95b4dbabe84df982421ad8afcbf6ff P abb474f603a9490e82a2f69e9f2e3403: Deleting consensus metadata
I20260811 00:20:00.026088 30805 tablet_bootstrap.cc:492] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: No bootstrap required, opened a new log
I20260811 00:20:00.026253 30805 ts_tablet_manager.cc:1397] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.012s
I20260811 00:20:00.027236 30805 raft_consensus.cc:348] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.027485 30805 raft_consensus.cc:374] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.027597 30805 raft_consensus.cc:729] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb474f603a9490e82a2f69e9f2e3403, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.027901 29860 catalog_manager.cc:4841] TS abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065): tablet ce95b4dbabe84df982421ad8afcbf6ff (table test-table [id=ad74f6d07efe4664ae590df23cb7b73e]) successfully deleted
I20260811 00:20:00.028060 30805 consensus_queue.cc:260] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.029265 30805 ts_tablet_manager.cc:1428] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 00:20:00.029400 30860 ts_tablet_manager.cc:1918] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:00.029627 30860 log.cc:1196] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/ce95b4dbabe84df982421ad8afcbf6ff
I20260811 00:20:00.030143 30860 ts_tablet_manager.cc:1939] T ce95b4dbabe84df982421ad8afcbf6ff P 8cccde774423488695a74daefce09560: Deleting consensus metadata
I20260811 00:20:00.032385 29858 catalog_manager.cc:4841] TS 8cccde774423488695a74daefce09560 (127.16.71.130:34911): tablet ce95b4dbabe84df982421ad8afcbf6ff (table test-table [id=ad74f6d07efe4664ae590df23cb7b73e]) successfully deleted
I20260811 00:20:00.246328 30850 raft_consensus.cc:482] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:20:00.246592 30850 raft_consensus.cc:504] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.247514 30850 leader_election.cc:288] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:20:00.248202 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11283776d96a4e5590fcf7d138fff942" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460" is_pre_election: true
I20260811 00:20:00.248299 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11283776d96a4e5590fcf7d138fff942" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403" is_pre_election: true
I20260811 00:20:00.248462 30166 raft_consensus.cc:2454] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 0.
I20260811 00:20:00.248653 30679 raft_consensus.cc:2454] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 0.
I20260811 00:20:00.249043 30299 leader_election.cc:302] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, d07358696fb3463a9e0e844e6b338460; no voters:
I20260811 00:20:00.249380 30850 raft_consensus.cc:2781] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:20:00.249524 30850 raft_consensus.cc:482] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:20:00.249634 30850 raft_consensus.cc:3037] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.252044 30850 raft_consensus.cc:504] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.252718 30850 leader_election.cc:288] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 election: Requested vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:20:00.253441 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11283776d96a4e5590fcf7d138fff942" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460"
I20260811 00:20:00.253523 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11283776d96a4e5590fcf7d138fff942" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403"
I20260811 00:20:00.253665 30166 raft_consensus.cc:3037] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.253787 30679 raft_consensus.cc:3037] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.256425 30166 raft_consensus.cc:2454] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 1.
I20260811 00:20:00.256956 30299 leader_election.cc:302] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, d07358696fb3463a9e0e844e6b338460; no voters:
I20260811 00:20:00.257232 30679 raft_consensus.cc:2454] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 1.
I20260811 00:20:00.257452 30850 raft_consensus.cc:2781] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:00.257704 30850 raft_consensus.cc:686] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 1 LEADER]: Becoming Leader. State: Replica: 8cccde774423488695a74daefce09560, State: Running, Role: LEADER
I20260811 00:20:00.258059 30850 consensus_queue.cc:237] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.261168 29878 catalog_manager.cc:5495] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cccde774423488695a74daefce09560 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:20:00 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407596, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.190@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:20:00.330876 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.315245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34609 (local address 127.0.0.1:60618)
0811 00:20:00.315975 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:00.316009 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:00.316193 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:00.316587 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:00.316592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:00.317416 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:00.317671 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.317683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.318823 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.318827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.319835 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.319849 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.320666 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.320673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.320841 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.321665 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:00.321685 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:00.324401 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:00.328033 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:00.328054 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:00.328065 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:00.328505 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:00.328895 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:00.328900 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:00.328903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:00.329053 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:00.329738 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:00.329745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:00.329902 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 00:20:00.330063 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 00:20:00.330443 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":383,"threads_started":1}
I20260811 00:20:00.330950 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.315278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.16.71.190:34609)
0811 00:20:00.316296 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.316301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.316327 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:20:00.316379 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.316383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.316438 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.316530 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:00.317870 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.318667 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.320002 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.320439 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.320555 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.324707 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:00.324713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:00.324717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:00.324753 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:00.327835 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:00.328659 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:00.328662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:00.328665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:00.328782 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:00.329165 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:00.329170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:00.329172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:00.329515 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:00.329670 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:00.330212 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 00:20:00.330596 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":163,"server-negotiator.queue_time_us":904,"thread_start_us":683,"threads_started":1}
I20260811 00:20:00.337306 29878 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:60618:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:20:00.338682 29878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:20:00.346396 30123 tablet_service.cc:1467] Processing CreateTablet for tablet ad97b22be08142509f0199ccb84901f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd71059ab28342ed9c217f8c75a6afd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:20:00.346710 30637 tablet_service.cc:1467] Processing CreateTablet for tablet ad97b22be08142509f0199ccb84901f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd71059ab28342ed9c217f8c75a6afd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:20:00.347110 30123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad97b22be08142509f0199ccb84901f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.347254 30637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad97b22be08142509f0199ccb84901f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.354187 30394 tablet_service.cc:1467] Processing CreateTablet for tablet ad97b22be08142509f0199ccb84901f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd71059ab28342ed9c217f8c75a6afd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:20:00.354925 30394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad97b22be08142509f0199ccb84901f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.356171 30805 tablet_bootstrap.cc:492] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: Bootstrap starting.
I20260811 00:20:00.357914 30805 tablet_bootstrap.cc:654] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.358912 30803 tablet_bootstrap.cc:492] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: Bootstrap starting.
I20260811 00:20:00.360317 30803 tablet_bootstrap.cc:654] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.362640 30803 tablet_bootstrap.cc:492] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: No bootstrap required, opened a new log
I20260811 00:20:00.362814 30803 ts_tablet_manager.cc:1397] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 00:20:00.363240 30804 tablet_bootstrap.cc:492] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: Bootstrap starting.
I20260811 00:20:00.363632 30803 raft_consensus.cc:348] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.363885 30803 raft_consensus.cc:374] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.363960 30803 raft_consensus.cc:729] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07358696fb3463a9e0e844e6b338460, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.364212 30803 consensus_queue.cc:260] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.365208 30804 tablet_bootstrap.cc:654] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.364984 30832 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.365358 30832 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:20:00.365856 30803 ts_tablet_manager.cc:1428] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:20:00.367583 30805 tablet_bootstrap.cc:492] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: No bootstrap required, opened a new log
I20260811 00:20:00.367777 30805 ts_tablet_manager.cc:1397] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260811 00:20:00.368650 30805 raft_consensus.cc:348] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.369329 30805 raft_consensus.cc:374] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.369483 30805 raft_consensus.cc:729] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb474f603a9490e82a2f69e9f2e3403, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.369784 30805 consensus_queue.cc:260] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.370483 30805 ts_tablet_manager.cc:1428] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:20:00.370484 30828 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.371384 30828 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:20:00.371663 30804 tablet_bootstrap.cc:492] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: No bootstrap required, opened a new log
I20260811 00:20:00.371837 30804 ts_tablet_manager.cc:1397] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260811 00:20:00.372761 30804 raft_consensus.cc:348] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.373034 30804 raft_consensus.cc:374] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.373143 30804 raft_consensus.cc:729] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cccde774423488695a74daefce09560, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.373517 30804 consensus_queue.cc:260] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.374250 30804 ts_tablet_manager.cc:1428] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:20:00.374118 30850 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.374505 30850 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:20:00.427093 30850 raft_consensus.cc:482] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:20:00.427534 30850 raft_consensus.cc:504] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.428303 30850 leader_election.cc:288] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:20:00.429111 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad97b22be08142509f0199ccb84901f5" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460" is_pre_election: true
I20260811 00:20:00.429183 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad97b22be08142509f0199ccb84901f5" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403" is_pre_election: true
I20260811 00:20:00.429574 30166 raft_consensus.cc:2454] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 0.
I20260811 00:20:00.429606 30679 raft_consensus.cc:2454] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 0.
I20260811 00:20:00.430217 30299 leader_election.cc:302] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, abb474f603a9490e82a2f69e9f2e3403; no voters:
I20260811 00:20:00.430635 30850 raft_consensus.cc:2781] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:20:00.430765 30850 raft_consensus.cc:482] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:20:00.430874 30850 raft_consensus.cc:3037] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.433804 30850 raft_consensus.cc:504] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.434451 30850 leader_election.cc:288] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 election: Requested vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:20:00.435168 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad97b22be08142509f0199ccb84901f5" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460"
I20260811 00:20:00.435312 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad97b22be08142509f0199ccb84901f5" candidate_uuid: "8cccde774423488695a74daefce09560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403"
I20260811 00:20:00.435526 30166 raft_consensus.cc:3037] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.435595 30679 raft_consensus.cc:3037] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.439041 30166 raft_consensus.cc:2454] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 1.
I20260811 00:20:00.439041 30679 raft_consensus.cc:2454] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cccde774423488695a74daefce09560 in term 1.
I20260811 00:20:00.439657 30299 leader_election.cc:302] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, d07358696fb3463a9e0e844e6b338460; no voters:
I20260811 00:20:00.440021 30850 raft_consensus.cc:2781] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:00.440280 30850 raft_consensus.cc:686] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [term 1 LEADER]: Becoming Leader. State: Replica: 8cccde774423488695a74daefce09560, State: Running, Role: LEADER
I20260811 00:20:00.440647 30850 consensus_queue.cc:237] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.442440 30835 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: TxnStatusTablet state changed. Reason: New leader 8cccde774423488695a74daefce09560. Latest consensus state: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.442764 30835 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:20:00.443519 30897 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:20:00.443754 30897 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:20:00.444319 29881 catalog_manager.cc:5495] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cccde774423488695a74daefce09560 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } health_report { overall_health: UNKNOWN } } }
I20260811 00:20:00.468906 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.463499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34911 (local address 127.0.0.1:35926)
0811 00:20:00.463755 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:00.463776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:00.463953 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:00.464492 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:00.464497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:00.464510 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:20:00.464732 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.464743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.465837 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.465842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.466852 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.466864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.467628 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.467635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.467831 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.467901 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:20:00.468310 (+ 409us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:20:00.468333 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:20:00.468405 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 00:20:00.468574 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 00:20:00.468951 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.463485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.16.71.130:34911)
0811 00:20:00.464191 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.464195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.464210 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:20:00.464278 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.464281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.464345 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.464443 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:20:00.464879 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.465684 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.467008 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.467475 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.467606 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.468099 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:20:00.468211 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:20:00.468288 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:00.468460 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 00:20:00.468548 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":372,"threads_started":1}
I20260811 00:20:00.473315 30679 raft_consensus.cc:1261] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Refusing update from remote peer 8cccde774423488695a74daefce09560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:00.473448 30166 raft_consensus.cc:1261] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Refusing update from remote peer 8cccde774423488695a74daefce09560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:00.474042 30850 consensus_queue.cc:1035] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:00.474334 30835 consensus_queue.cc:1035] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:00.484442 30828 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: TxnStatusTablet state changed. Reason: New leader 8cccde774423488695a74daefce09560. Latest consensus state: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.484781 30828 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:20:00.489120 30901 mvcc.cc:205] Tried to move back new op lower bound from 7317125531532369920 to 7317125531407581184. Current Snapshot: MvccSnapshot[applied={T|T < 7317125531532369920}]
I20260811 00:20:00.490687 30905 mvcc.cc:205] Tried to move back new op lower bound from 7317125531532369920 to 7317125531407581184. Current Snapshot: MvccSnapshot[applied={T|T < 7317125531532369920}]
I20260811 00:20:00.495649 30832 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: TxnStatusTablet state changed. Reason: New leader 8cccde774423488695a74daefce09560. Latest consensus state: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.495965 30832 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:20:00.508487 30850 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.508912 30850 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:20:00.509420 30850 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.509651 30850 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P 8cccde774423488695a74daefce09560: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:20:00.514573 30828 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.514902 30828 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P abb474f603a9490e82a2f69e9f2e3403: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:20:00.518059 30832 tablet_replica.cc:439] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cccde774423488695a74daefce09560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } } }
I20260811 00:20:00.518349 30832 tablet_replica.cc:442] T ad97b22be08142509f0199ccb84901f5 P d07358696fb3463a9e0e844e6b338460: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:20:00.543800 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.538139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:58907 (local address 127.16.71.130:34911)
0811 00:20:00.538306 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.538310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.538895 (+ 585us) server_negotiation.cc:407] Connection header received
0811 00:20:00.539101 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.539105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.539168 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.539239 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:20:00.539724 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.540514 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.541956 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.542412 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.542527 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.543015 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:20:00.543131 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:20:00.543203 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:00.543334 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 00:20:00.543439 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260811 00:20:00.543886 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.538181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34911 (local address 127.16.71.130:58907)
0811 00:20:00.538777 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:00.538798 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:00.538964 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:00.539276 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:00.539280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:00.539291 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:20:00.539541 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.539550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.540686 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.540690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.541812 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.541825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.542608 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.542614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.542745 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.542813 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:20:00.543222 (+ 409us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:20:00.543233 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:20:00.543309 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 00:20:00.543479 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":283,"threads_started":1}
I20260811 00:20:00.554435 30166 raft_consensus.cc:1261] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Refusing update from remote peer 8cccde774423488695a74daefce09560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:00.554503 30678 raft_consensus.cc:1261] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Refusing update from remote peer 8cccde774423488695a74daefce09560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:00.555073 30835 consensus_queue.cc:1035] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:00.555557 30835 consensus_queue.cc:1035] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:00.561062 30904 mvcc.cc:205] Tried to move back new op lower bound from 7317125531865677824 to 7317125530659590144. Current Snapshot: MvccSnapshot[applied={T|T < 7317125531865677824}]
I20260811 00:20:00.630033 29881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:00.630275 29881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:00.634087 29881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Sending DeleteTablet for 3 replicas of tablet 11283776d96a4e5590fcf7d138fff942
I20260811 00:20:00.635183 30392 tablet_service.cc:1514] Processing DeleteTablet for tablet 11283776d96a4e5590fcf7d138fff942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:00 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:35912
I20260811 00:20:00.635488 30637 tablet_service.cc:1514] Processing DeleteTablet for tablet 11283776d96a4e5590fcf7d138fff942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:00 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:58162
I20260811 00:20:00.636099 30920 tablet_replica.cc:330] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: stopping tablet replica
I20260811 00:20:00.636605 30920 raft_consensus.cc:2229] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:20:00.637246 30920 raft_consensus.cc:2258] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:00.641134 30123 tablet_service.cc:1514] Processing DeleteTablet for tablet 11283776d96a4e5590fcf7d138fff942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:00 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:39146
I20260811 00:20:00.641571 30920 ts_tablet_manager.cc:1905] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:00.649852 30920 ts_tablet_manager.cc:1918] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:20:00.650192 30920 log.cc:1196] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/11283776d96a4e5590fcf7d138fff942
I20260811 00:20:00.650846 30920 ts_tablet_manager.cc:1939] T 11283776d96a4e5590fcf7d138fff942 P 8cccde774423488695a74daefce09560: Deleting consensus metadata
I20260811 00:20:00.652477 29858 catalog_manager.cc:4841] TS 8cccde774423488695a74daefce09560 (127.16.71.130:34911): tablet 11283776d96a4e5590fcf7d138fff942 (table test-table [id=dbd1132b3fb64b42ac2eca427e085efa]) successfully deleted
I20260811 00:20:00.655341 30923 tablet_replica.cc:330] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: stopping tablet replica
I20260811 00:20:00.655627 30923 raft_consensus.cc:2229] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:00.655921 30923 raft_consensus.cc:2258] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:00.657327 30921 tablet_replica.cc:330] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: stopping tablet replica
I20260811 00:20:00.657570 30921 raft_consensus.cc:2229] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:00.657874 30921 raft_consensus.cc:2258] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:00.659173 30921 ts_tablet_manager.cc:1905] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:00.661787 30923 ts_tablet_manager.cc:1905] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:00.665105 30921 ts_tablet_manager.cc:1918] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:20:00.665253 30921 log.cc:1196] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/11283776d96a4e5590fcf7d138fff942
I20260811 00:20:00.665656 30921 ts_tablet_manager.cc:1939] T 11283776d96a4e5590fcf7d138fff942 P abb474f603a9490e82a2f69e9f2e3403: Deleting consensus metadata
I20260811 00:20:00.667022 29860 catalog_manager.cc:4841] TS abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065): tablet 11283776d96a4e5590fcf7d138fff942 (table test-table [id=dbd1132b3fb64b42ac2eca427e085efa]) successfully deleted
I20260811 00:20:00.669597 30923 ts_tablet_manager.cc:1918] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:20:00.669780 30923 log.cc:1196] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/11283776d96a4e5590fcf7d138fff942
I20260811 00:20:00.670363 30923 ts_tablet_manager.cc:1939] T 11283776d96a4e5590fcf7d138fff942 P d07358696fb3463a9e0e844e6b338460: Deleting consensus metadata
I20260811 00:20:00.671993 29860 catalog_manager.cc:4841] TS d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207): tablet 11283776d96a4e5590fcf7d138fff942 (table test-table [id=dbd1132b3fb64b42ac2eca427e085efa]) successfully deleted
Aug 11 00:20:00 dist-test-slave-f7th krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:20:00 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:00 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:20:00.725270 30886 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:00.712144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58820 (local address 127.16.71.190:34609)
0811 00:20:00.712384 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.712390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.712745 (+ 355us) server_negotiation.cc:407] Connection header received
0811 00:20:00.712929 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.712933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.713034 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.713128 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:00.714835 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.715675 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.717604 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.718103 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.718219 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.725004 (+ 6785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58820: BlockingRecv error: recv got EOF from 127.0.0.1:58820 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 00:20:00 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM
I20260811 00:20:00.745699 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.728310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.16.71.190:34609)
0811 00:20:00.728545 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.728553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.729119 (+ 566us) server_negotiation.cc:407] Connection header received
0811 00:20:00.729266 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.729273 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.729362 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.729480 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:00.730822 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.732082 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.733589 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.734267 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.734427 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.739093 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:00.739099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:00.739103 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:00.739147 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:00.741886 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:00.743044 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:00.743050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:00.743052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:00.743224 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:00.743942 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:00.743947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:00.743950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:00.744288 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:00.744538 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:00.745087 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 00:20:00.745314 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:20:00.750413 29881 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:20:00.751212 29881 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:20:00.759717 30123 tablet_service.cc:1467] Processing CreateTablet for tablet 85c9eb7aa24543f8acec2bccc885f36f (DEFAULT_TABLE table=test-table [id=e7578700a1f343b6bd301011c234962b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:00.759714 30392 tablet_service.cc:1467] Processing CreateTablet for tablet 85c9eb7aa24543f8acec2bccc885f36f (DEFAULT_TABLE table=test-table [id=e7578700a1f343b6bd301011c234962b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:00.760200 30123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c9eb7aa24543f8acec2bccc885f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.760371 30392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c9eb7aa24543f8acec2bccc885f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.763674 30637 tablet_service.cc:1467] Processing CreateTablet for tablet 85c9eb7aa24543f8acec2bccc885f36f (DEFAULT_TABLE table=test-table [id=e7578700a1f343b6bd301011c234962b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:00.764190 30637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c9eb7aa24543f8acec2bccc885f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.767977 30804 tablet_bootstrap.cc:492] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: Bootstrap starting.
I20260811 00:20:00.771451 30804 tablet_bootstrap.cc:654] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.772270 30803 tablet_bootstrap.cc:492] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: Bootstrap starting.
I20260811 00:20:00.774037 30803 tablet_bootstrap.cc:654] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.775193 30805 tablet_bootstrap.cc:492] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: Bootstrap starting.
I20260811 00:20:00.777340 30805 tablet_bootstrap.cc:654] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.779881 30803 tablet_bootstrap.cc:492] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: No bootstrap required, opened a new log
I20260811 00:20:00.780082 30803 ts_tablet_manager.cc:1397] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260811 00:20:00.782436 30803 raft_consensus.cc:348] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.782750 30803 raft_consensus.cc:374] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.782851 30803 raft_consensus.cc:729] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07358696fb3463a9e0e844e6b338460, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.783113 30803 consensus_queue.cc:260] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.787914 30804 tablet_bootstrap.cc:492] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: No bootstrap required, opened a new log
I20260811 00:20:00.787905 30803 ts_tablet_manager.cc:1428] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260811 00:20:00.788219 30804 ts_tablet_manager.cc:1397] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.008s
I20260811 00:20:00.789361 30804 raft_consensus.cc:348] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.789786 30804 raft_consensus.cc:374] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.790041 30804 raft_consensus.cc:729] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cccde774423488695a74daefce09560, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.790385 30804 consensus_queue.cc:260] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.791131 30804 ts_tablet_manager.cc:1428] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:20:00.791353 30805 tablet_bootstrap.cc:492] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: No bootstrap required, opened a new log
I20260811 00:20:00.791535 30805 ts_tablet_manager.cc:1397] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260811 00:20:00.792433 30805 raft_consensus.cc:348] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.792733 30805 raft_consensus.cc:374] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.792833 30805 raft_consensus.cc:729] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb474f603a9490e82a2f69e9f2e3403, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.793069 30805 consensus_queue.cc:260] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.793627 30805 ts_tablet_manager.cc:1428] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:20:00.810391 30828 raft_consensus.cc:482] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:20:00.810660 30828 raft_consensus.cc:504] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.812424 30828 leader_election.cc:288] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), 8cccde774423488695a74daefce09560 (127.16.71.130:34911)
I20260811 00:20:00.834967 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.813029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:45737 (local address 127.16.71.129:46207)
0811 00:20:00.827885 (+ 14856us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.827892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.827921 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:20:00.828069 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.828075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.828155 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.828291 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:00.829063 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.830345 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.833060 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.833912 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.834125 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.834208 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:00.834508 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 00:20:00.834647 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14706,"thread_start_us":13250,"threads_started":1}
I20260811 00:20:00.835297 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.812886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46207 (local address 127.16.71.131:45737)
0811 00:20:00.826796 (+ 13910us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:00.826845 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:00.827076 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:00.828333 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:00.828338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:00.828364 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:00.828822 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.828836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.830491 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.830496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.832816 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.832838 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.834129 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.834136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.834313 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.834342 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 00:20:00.834425 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 00:20:00.834575 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13667,"mutex_wait_us":39,"thread_start_us":14493,"threads_started":1}
I20260811 00:20:00.835840 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.813511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:43615 (local address 127.16.71.130:34911)
0811 00:20:00.813745 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.813752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.828014 (+ 14262us) server_negotiation.cc:407] Connection header received
0811 00:20:00.828320 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.828326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.828418 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.828535 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:00.829206 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.830034 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.833038 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.834418 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.834600 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.834677 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:00.835286 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 00:20:00.835462 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:20:00.836077 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c9eb7aa24543f8acec2bccc885f36f" candidate_uuid: "abb474f603a9490e82a2f69e9f2e3403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460" is_pre_election: true
I20260811 00:20:00.836338 30166 raft_consensus.cc:2454] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb474f603a9490e82a2f69e9f2e3403 in term 0.
I20260811 00:20:00.836794 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.812952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34911 (local address 127.16.71.131:43615)
0811 00:20:00.827874 (+ 14922us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:00.827894 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:00.828133 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:00.828574 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:00.828578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:00.828601 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:00.829047 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.829060 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.830184 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.830189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.832820 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:00.832837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.834865 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.834872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:00.835073 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.835109 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:20:00.836158 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0811 00:20:00.836325 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14736,"mutex_wait_us":107,"thread_start_us":12706,"threads_started":1}
I20260811 00:20:00.836943 30560 leader_election.cc:302] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abb474f603a9490e82a2f69e9f2e3403, d07358696fb3463a9e0e844e6b338460; no voters:
I20260811 00:20:00.837447 30828 raft_consensus.cc:2781] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:20:00.837590 30828 raft_consensus.cc:482] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:20:00.837689 30828 raft_consensus.cc:3037] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.837786 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c9eb7aa24543f8acec2bccc885f36f" candidate_uuid: "abb474f603a9490e82a2f69e9f2e3403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cccde774423488695a74daefce09560" is_pre_election: true
I20260811 00:20:00.838312 30432 raft_consensus.cc:2454] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb474f603a9490e82a2f69e9f2e3403 in term 0.
I20260811 00:20:00.841229 30828 raft_consensus.cc:504] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.842046 30828 leader_election.cc:288] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [CANDIDATE]: Term 1 election: Requested vote from peers d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207), 8cccde774423488695a74daefce09560 (127.16.71.130:34911)
I20260811 00:20:00.842666 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c9eb7aa24543f8acec2bccc885f36f" candidate_uuid: "abb474f603a9490e82a2f69e9f2e3403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cccde774423488695a74daefce09560"
I20260811 00:20:00.842666 30166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c9eb7aa24543f8acec2bccc885f36f" candidate_uuid: "abb474f603a9490e82a2f69e9f2e3403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07358696fb3463a9e0e844e6b338460"
I20260811 00:20:00.842906 30432 raft_consensus.cc:3037] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.842923 30166 raft_consensus.cc:3037] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:00.845505 30432 raft_consensus.cc:2454] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb474f603a9490e82a2f69e9f2e3403 in term 1.
I20260811 00:20:00.846107 30166 raft_consensus.cc:2454] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb474f603a9490e82a2f69e9f2e3403 in term 1.
I20260811 00:20:00.846117 30557 leader_election.cc:302] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, abb474f603a9490e82a2f69e9f2e3403; no voters:
I20260811 00:20:00.846439 30828 raft_consensus.cc:2781] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:00.846974 30828 raft_consensus.cc:686] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 1 LEADER]: Becoming Leader. State: Replica: abb474f603a9490e82a2f69e9f2e3403, State: Running, Role: LEADER
I20260811 00:20:00.847425 30828 consensus_queue.cc:237] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.851540 29881 catalog_manager.cc:5495] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 reported cstate change: term changed from 0 to 1, leader changed from <none> to abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131). New cstate: current_term: 1 leader_uuid: "abb474f603a9490e82a2f69e9f2e3403" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } health_report { overall_health: HEALTHY } } }
I20260811 00:20:00.869289 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:00.863175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.16.71.131:41065)
0811 00:20:00.863787 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:00.863791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:00.863811 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:20:00.863904 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:00.863907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:00.864000 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:00.864154 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:20:00.864579 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.865421 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.866745 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:00.867207 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:00.867327 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:00.867834 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:20:00.868519 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:20:00.868693 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:00.868845 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 00:20:00.868939 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":335,"threads_started":1}
I20260811 00:20:00.873628 30166 raft_consensus.cc:1261] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Refusing update from remote peer abb474f603a9490e82a2f69e9f2e3403: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:00.873878 30432 raft_consensus.cc:1261] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Refusing update from remote peer abb474f603a9490e82a2f69e9f2e3403: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:00.874565 30956 consensus_queue.cc:1035] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:00.874994 30828 consensus_queue.cc:1035] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:00.901523 29881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:00.901829 29881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:00.905829 29881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Sending DeleteTablet for 3 replicas of tablet 85c9eb7aa24543f8acec2bccc885f36f
I20260811 00:20:00.906906 30123 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c9eb7aa24543f8acec2bccc885f36f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:00 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:39146
I20260811 00:20:00.907303 30392 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c9eb7aa24543f8acec2bccc885f36f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:00 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:35912
I20260811 00:20:00.907625 30923 tablet_replica.cc:330] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: stopping tablet replica
I20260811 00:20:00.907858 30923 raft_consensus.cc:2229] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:00.908143 30923 raft_consensus.cc:2258] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:00.908478 30920 tablet_replica.cc:330] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: stopping tablet replica
I20260811 00:20:00.908731 30920 raft_consensus.cc:2229] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:00.909020 30920 raft_consensus.cc:2258] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:00.910137 30637 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c9eb7aa24543f8acec2bccc885f36f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:00 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:58162
I20260811 00:20:00.910475 30921 tablet_replica.cc:330] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: stopping tablet replica
I20260811 00:20:00.910691 30921 raft_consensus.cc:2229] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:20:00.911160 30920 ts_tablet_manager.cc:1905] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:00.911319 30921 raft_consensus.cc:2258] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:00.912788 30921 ts_tablet_manager.cc:1905] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:00.918677 30920 ts_tablet_manager.cc:1918] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:00.918864 30920 log.cc:1196] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/85c9eb7aa24543f8acec2bccc885f36f
I20260811 00:20:00.919365 30920 ts_tablet_manager.cc:1939] T 85c9eb7aa24543f8acec2bccc885f36f P 8cccde774423488695a74daefce09560: Deleting consensus metadata
I20260811 00:20:00.919905 30921 ts_tablet_manager.cc:1918] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:00.920068 30921 log.cc:1196] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/85c9eb7aa24543f8acec2bccc885f36f
I20260811 00:20:00.920538 30921 ts_tablet_manager.cc:1939] T 85c9eb7aa24543f8acec2bccc885f36f P abb474f603a9490e82a2f69e9f2e3403: Deleting consensus metadata
I20260811 00:20:00.920856 29858 catalog_manager.cc:4841] TS 8cccde774423488695a74daefce09560 (127.16.71.130:34911): tablet 85c9eb7aa24543f8acec2bccc885f36f (table test-table [id=e7578700a1f343b6bd301011c234962b]) successfully deleted
I20260811 00:20:00.921228 29881 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 00:20:00.921999 30923 ts_tablet_manager.cc:1905] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 00:20:00.922000 29881 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:20:00.927726 29860 catalog_manager.cc:4841] TS abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065): tablet 85c9eb7aa24543f8acec2bccc885f36f (table test-table [id=e7578700a1f343b6bd301011c234962b]) successfully deleted
I20260811 00:20:00.928417 30923 ts_tablet_manager.cc:1918] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:00.928808 30923 log.cc:1196] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/85c9eb7aa24543f8acec2bccc885f36f
I20260811 00:20:00.929473 30923 ts_tablet_manager.cc:1939] T 85c9eb7aa24543f8acec2bccc885f36f P d07358696fb3463a9e0e844e6b338460: Deleting consensus metadata
I20260811 00:20:00.930845 29860 catalog_manager.cc:4841] TS d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207): tablet 85c9eb7aa24543f8acec2bccc885f36f (table test-table [id=e7578700a1f343b6bd301011c234962b]) successfully deleted
I20260811 00:20:00.932020 30123 tablet_service.cc:1467] Processing CreateTablet for tablet 31ddc4175c0e4947932668c4f193827c (DEFAULT_TABLE table=test-table [id=93465ee415a8493094922e7e8504d376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:00.932206 30392 tablet_service.cc:1467] Processing CreateTablet for tablet 31ddc4175c0e4947932668c4f193827c (DEFAULT_TABLE table=test-table [id=93465ee415a8493094922e7e8504d376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:00.932549 30123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ddc4175c0e4947932668c4f193827c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.932687 30392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ddc4175c0e4947932668c4f193827c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.937458 30637 tablet_service.cc:1467] Processing CreateTablet for tablet 31ddc4175c0e4947932668c4f193827c (DEFAULT_TABLE table=test-table [id=93465ee415a8493094922e7e8504d376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:00.937981 30637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ddc4175c0e4947932668c4f193827c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:00.940696 30803 tablet_bootstrap.cc:492] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: Bootstrap starting.
I20260811 00:20:00.942479 30803 tablet_bootstrap.cc:654] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.952536 30804 tablet_bootstrap.cc:492] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: Bootstrap starting.
I20260811 00:20:00.954438 30805 tablet_bootstrap.cc:492] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: Bootstrap starting.
I20260811 00:20:00.954900 30804 tablet_bootstrap.cc:654] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.959667 30805 tablet_bootstrap.cc:654] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:00.973500 30803 tablet_bootstrap.cc:492] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: No bootstrap required, opened a new log
I20260811 00:20:00.973757 30803 ts_tablet_manager.cc:1397] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.005s
I20260811 00:20:00.976416 30803 raft_consensus.cc:348] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.976747 30803 raft_consensus.cc:374] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.976836 30803 raft_consensus.cc:729] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07358696fb3463a9e0e844e6b338460, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.977216 30803 consensus_queue.cc:260] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.977941 30804 tablet_bootstrap.cc:492] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: No bootstrap required, opened a new log
I20260811 00:20:00.978127 30804 ts_tablet_manager.cc:1397] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.000s
I20260811 00:20:00.978981 30804 raft_consensus.cc:348] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.979277 30804 raft_consensus.cc:374] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.979367 30804 raft_consensus.cc:729] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cccde774423488695a74daefce09560, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.979668 30804 consensus_queue.cc:260] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.981065 30803 ts_tablet_manager.cc:1428] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 00:20:00.981490 30804 ts_tablet_manager.cc:1428] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 00:20:00.984922 30805 tablet_bootstrap.cc:492] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: No bootstrap required, opened a new log
I20260811 00:20:00.985291 30805 ts_tablet_manager.cc:1397] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.003s
I20260811 00:20:00.986346 30805 raft_consensus.cc:348] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.986781 30805 raft_consensus.cc:374] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:00.986944 30805 raft_consensus.cc:729] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb474f603a9490e82a2f69e9f2e3403, State: Initialized, Role: FOLLOWER
I20260811 00:20:00.987358 30805 consensus_queue.cc:260] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:00.988075 30805 ts_tablet_manager.cc:1428] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:20:01.065129 30832 raft_consensus.cc:482] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:20:01.065374 30832 raft_consensus.cc:504] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:01.067096 30832 leader_election.cc:288] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cccde774423488695a74daefce09560 (127.16.71.130:34911), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:20:01.076526 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.067559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:57249 (local address 127.16.71.131:41065)
0811 00:20:01.067768 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.067775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.068427 (+ 652us) server_negotiation.cc:407] Connection header received
0811 00:20:01.068737 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.068744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.068835 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.068906 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:01.069813 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.071091 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.074267 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.075360 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.075545 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.075638 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.076043 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.076210 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:20:01.076749 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.067449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:41065 (local address 127.16.71.129:57249)
0811 00:20:01.068301 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:01.068320 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:01.068492 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:01.069164 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:01.069169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:01.069200 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:01.069600 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.069615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.071302 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.071307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.073982 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.074013 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.075645 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.075653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.075846 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.075875 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 00:20:01.076058 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 00:20:01.076279 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":69,"thread_start_us":545,"threads_started":1}
I20260811 00:20:01.077072 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.067563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:48775 (local address 127.16.71.130:34911)
0811 00:20:01.067762 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.067768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.069047 (+ 1279us) server_negotiation.cc:407] Connection header received
0811 00:20:01.069324 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.069330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.069406 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.069470 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:01.070289 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.071283 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.074247 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.075568 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.075805 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.075889 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.076544 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.076753 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:20:01.077486 30678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ddc4175c0e4947932668c4f193827c" candidate_uuid: "d07358696fb3463a9e0e844e6b338460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403" is_pre_election: true
I20260811 00:20:01.077906 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.068217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34911 (local address 127.16.71.129:48775)
0811 00:20:01.068944 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:01.068962 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:01.069154 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:01.069684 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:01.069688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:01.069714 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:01.070073 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.070086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.071463 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.071467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.073982 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.074011 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.076107 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.076114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.076314 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.076353 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:20:01.077333 (+ 980us) client_negotiation.cc:240] Negotiation successful
0811 00:20:01.077497 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":44,"thread_start_us":3044,"threads_started":1}
I20260811 00:20:01.077883 30678 raft_consensus.cc:2454] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07358696fb3463a9e0e844e6b338460 in term 0.
I20260811 00:20:01.078534 30045 leader_election.cc:302] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abb474f603a9490e82a2f69e9f2e3403, d07358696fb3463a9e0e844e6b338460; no voters:
I20260811 00:20:01.078724 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ddc4175c0e4947932668c4f193827c" candidate_uuid: "d07358696fb3463a9e0e844e6b338460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cccde774423488695a74daefce09560" is_pre_election: true
I20260811 00:20:01.079000 30832 raft_consensus.cc:2781] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:20:01.079017 30432 raft_consensus.cc:2454] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07358696fb3463a9e0e844e6b338460 in term 0.
I20260811 00:20:01.079101 30832 raft_consensus.cc:482] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:20:01.079162 30832 raft_consensus.cc:3037] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:01.081676 30832 raft_consensus.cc:504] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:01.082363 30832 leader_election.cc:288] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [CANDIDATE]: Term 1 election: Requested vote from peers 8cccde774423488695a74daefce09560 (127.16.71.130:34911), abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065)
I20260811 00:20:01.083060 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ddc4175c0e4947932668c4f193827c" candidate_uuid: "d07358696fb3463a9e0e844e6b338460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cccde774423488695a74daefce09560"
I20260811 00:20:01.083284 30432 raft_consensus.cc:3037] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:01.084928 30678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ddc4175c0e4947932668c4f193827c" candidate_uuid: "d07358696fb3463a9e0e844e6b338460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb474f603a9490e82a2f69e9f2e3403"
I20260811 00:20:01.085224 30678 raft_consensus.cc:3037] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:01.085848 30432 raft_consensus.cc:2454] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07358696fb3463a9e0e844e6b338460 in term 1.
I20260811 00:20:01.086459 30040 leader_election.cc:302] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cccde774423488695a74daefce09560, d07358696fb3463a9e0e844e6b338460; no voters:
I20260811 00:20:01.086865 30832 raft_consensus.cc:2781] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:01.087407 30832 raft_consensus.cc:686] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 1 LEADER]: Becoming Leader. State: Replica: d07358696fb3463a9e0e844e6b338460, State: Running, Role: LEADER
I20260811 00:20:01.087881 30832 consensus_queue.cc:237] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } }
I20260811 00:20:01.088486 30678 raft_consensus.cc:2454] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07358696fb3463a9e0e844e6b338460 in term 1.
I20260811 00:20:01.093474 29881 catalog_manager.cc:5495] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 reported cstate change: term changed from 0 to 1, leader changed from <none> to d07358696fb3463a9e0e844e6b338460 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "d07358696fb3463a9e0e844e6b338460" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07358696fb3463a9e0e844e6b338460" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 } health_report { overall_health: UNKNOWN } } }
I20260811 00:20:01.151830 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.144950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.16.71.129:46207)
0811 00:20:01.145152 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.145158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.145207 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:20:01.145574 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.145580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.145701 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.145891 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:20:01.146430 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.147509 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.149085 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.149705 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.149847 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.150351 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:20:01.151151 (+ 800us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:20:01.151314 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.151430 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.151534 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:20:01.171120 31002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.164452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34911 (local address 127.16.71.129:38371)
0811 00:20:01.165199 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:01.165216 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:01.165375 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:01.165732 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:01.165735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:01.165743 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:20:01.166042 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.166051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.167549 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.167554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.168853 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.168873 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.169908 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.169913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.170085 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.170156 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:20:01.170604 (+ 448us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:20:01.170612 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:20:01.170666 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 00:20:01.170786 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":378,"threads_started":1}
I20260811 00:20:01.171123 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.164502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:38371 (local address 127.16.71.130:34911)
0811 00:20:01.164734 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.164741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.165328 (+ 587us) server_negotiation.cc:407] Connection header received
0811 00:20:01.165506 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.165512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.165607 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.165728 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:20:01.166206 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.167339 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.169066 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.169719 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.169866 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.170350 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:20:01.170510 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:20:01.170611 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.170709 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.170807 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:20:01.182780 30432 raft_consensus.cc:1261] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Refusing update from remote peer d07358696fb3463a9e0e844e6b338460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:01.182819 30678 raft_consensus.cc:1261] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Refusing update from remote peer d07358696fb3463a9e0e844e6b338460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:01.183670 30984 consensus_queue.cc:1035] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb474f603a9490e82a2f69e9f2e3403" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:01.184193 30832 consensus_queue.cc:1035] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cccde774423488695a74daefce09560" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:01.240916 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.234275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:53701 (local address 127.16.71.129:46207)
0811 00:20:01.234487 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.234494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.235109 (+ 615us) server_negotiation.cc:407] Connection header received
0811 00:20:01.235325 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.235330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.235439 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.235598 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:20:01.236100 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.237027 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.238868 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.239562 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.239715 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.240083 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:20:01.240239 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:20:01.240335 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.240492 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.240582 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:20:01.241040 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.233828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46207 (local address 127.16.71.130:53701)
0811 00:20:01.234946 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:01.234967 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:01.235139 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:01.235637 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:01.235641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:01.235655 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:20:01.235892 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.235904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.237188 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.237192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.238689 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:01.238706 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.239726 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.239731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:01.239867 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.239911 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:20:01.240354 (+ 443us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:20:01.240366 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 00:20:01.240445 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 00:20:01.240566 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"thread_start_us":807,"threads_started":1}
I20260811 00:20:01.290532 29881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:01.290850 29881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:01.294960 29881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f53cfdfc642043dfa8aa4c39e6174312: Sending DeleteTablet for 3 replicas of tablet 31ddc4175c0e4947932668c4f193827c
I20260811 00:20:01.296129 30123 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ddc4175c0e4947932668c4f193827c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:01 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:39146
I20260811 00:20:01.296419 30392 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ddc4175c0e4947932668c4f193827c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:01 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:35912
I20260811 00:20:01.296496 30923 tablet_replica.cc:330] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: stopping tablet replica
I20260811 00:20:01.296728 30923 raft_consensus.cc:2229] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:20:01.296757 30920 tablet_replica.cc:330] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: stopping tablet replica
I20260811 00:20:01.296941 30920 raft_consensus.cc:2229] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:01.297176 30920 raft_consensus.cc:2258] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:01.297355 30923 raft_consensus.cc:2258] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:01.298480 30637 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ddc4175c0e4947932668c4f193827c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:01 UTC) from {username='oryx', principal='oryx/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:58162
I20260811 00:20:01.298820 30921 tablet_replica.cc:330] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: stopping tablet replica
I20260811 00:20:01.299028 30921 raft_consensus.cc:2229] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:01.299011 30920 ts_tablet_manager.cc:1905] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:01.299259 30921 raft_consensus.cc:2258] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:01.300541 30921 ts_tablet_manager.cc:1905] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:01.306008 30923 ts_tablet_manager.cc:1905] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:01.306744 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 30006
I20260811 00:20:01.314955 30923 ts_tablet_manager.cc:1918] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:20:01.315168 30923 log.cc:1196] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/31ddc4175c0e4947932668c4f193827c
I20260811 00:20:01.315649 30923 ts_tablet_manager.cc:1939] T 31ddc4175c0e4947932668c4f193827c P d07358696fb3463a9e0e844e6b338460: Deleting consensus metadata
I20260811 00:20:01.317298 30920 ts_tablet_manager.cc:1918] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:20:01.318040 30920 log.cc:1196] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/31ddc4175c0e4947932668c4f193827c
I20260811 00:20:01.318835 30920 ts_tablet_manager.cc:1939] T 31ddc4175c0e4947932668c4f193827c P 8cccde774423488695a74daefce09560: Deleting consensus metadata
I20260811 00:20:01.320832 29858 catalog_manager.cc:4841] TS 8cccde774423488695a74daefce09560 (127.16.71.130:34911): tablet 31ddc4175c0e4947932668c4f193827c (table test-table [id=93465ee415a8493094922e7e8504d376]) successfully deleted
I20260811 00:20:01.321302 30921 ts_tablet_manager.cc:1918] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:20:01.321491 30921 log.cc:1196] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/31ddc4175c0e4947932668c4f193827c
I20260811 00:20:01.321990 30921 ts_tablet_manager.cc:1939] T 31ddc4175c0e4947932668c4f193827c P abb474f603a9490e82a2f69e9f2e3403: Deleting consensus metadata
I20260811 00:20:01.323325 29860 catalog_manager.cc:4841] TS abb474f603a9490e82a2f69e9f2e3403 (127.16.71.131:41065): tablet 31ddc4175c0e4947932668c4f193827c (table test-table [id=93465ee415a8493094922e7e8504d376]) successfully deleted
I20260811 00:20:01.324038 29860 catalog_manager.cc:4841] TS d07358696fb3463a9e0e844e6b338460 (127.16.71.129:46207): tablet 31ddc4175c0e4947932668c4f193827c (table test-table [id=93465ee415a8493094922e7e8504d376]) successfully deleted
Aug 11 00:20:01 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.129@KRBTEST.COM
I20260811 00:20:01.345141 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.308865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.16.71.129:46207)
0811 00:20:01.309189 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.309196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.322903 (+ 13707us) server_negotiation.cc:407] Connection header received
0811 00:20:01.323459 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.323467 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.323576 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.323657 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:01.325745 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.326965 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.328985 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.329672 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.329831 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.336402 (+ 6571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:01.336431 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:01.336460 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:01.336518 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:01.340965 (+ 4447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:01.342105 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:01.342116 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:01.342120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:01.342307 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:01.342821 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:01.342832 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:01.342835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:01.343429 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:01.343746 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.344432 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.344760 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260811 00:20:01.346886 30244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:01.480284 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 30006
I20260811 00:20:01.512274 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 30258
Aug 11 00:20:01 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.130@KRBTEST.COM
I20260811 00:20:01.528548 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.513206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.16.71.130:34911)
0811 00:20:01.513390 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.513396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.513439 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:20:01.513491 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.513495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.513579 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.513664 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:01.515019 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.515720 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.517069 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.517520 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.517636 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.522545 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:01.522563 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:01.522582 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:01.522636 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:01.525530 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:01.526307 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:01.526316 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:01.526319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:01.526476 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:01.526870 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:01.526874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:01.526876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:01.527296 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:01.527617 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.527943 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.528199 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:20:01.529297 30512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:01.685333 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 30258
I20260811 00:20:01.729223 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 30533
Aug 11 00:20:01 dist-test-slave-f7th krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.71.131@KRBTEST.COM
I20260811 00:20:01.746722 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:01.730100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.16.71.131:41065)
0811 00:20:01.730626 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:01.730631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:01.730648 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:20:01.730705 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:01.730709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:01.730771 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:01.730867 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:01.732340 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.733210 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.734588 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:01.735030 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:01.735149 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:01.740069 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:01.740089 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:01.740109 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:01.740159 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:01.743476 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:01.744372 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:01.744381 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:01.744383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:01.744569 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:01.745031 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:01.745036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:01.745038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:01.745483 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:01.745761 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:01.746109 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 00:20:01.746331 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":266,"threads_started":1}
I20260811 00:20:01.747440 30756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:01.874349 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 30533
I20260811 00:20:01.907151 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 29827
I20260811 00:20:01.908545 29953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:02.029996 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 29827
2026-08-11T00:20:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7606 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:20:02 dist-test-slave-f7th krb5kdc[31029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:20:02 dist-test-slave-f7th krb5kdc[31029](info): set up 2 sockets
Aug 11 00:20:02 dist-test-slave-f7th krb5kdc[31029](info): commencing operation
krb5kdc: starting...
W20260811 00:20:03.359614 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:20:03 dist-test-slave-f7th krb5kdc[31029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:20:03.654491 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42175
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:34277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:42175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:04.030362 31064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:04.030750 31064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:04.030892 31064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:04.038645 31064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:20:04.038802 31064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:04.038877 31064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:04.038964 31064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:20:04.039032 31064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:20:04.050282 31064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:34277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:42175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:04.051781 31064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:04.053655 31064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:04.067054 31073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:04.067078 31074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:04.067163 31076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:04.068104 31064 server_base.cc:1034] running on GCE node
I20260811 00:20:04.069020 31064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:04.070710 31064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:04.071971 31064 hybrid_clock.cc:648] HybridClock initialized: now 1786407604071911 us; error 47 us; skew 500 ppm
I20260811 00:20:04.072398 31064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T00:20:04Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2101 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:20:04 dist-test-slave-f7th krb5kdc[31083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:20:04 dist-test-slave-f7th krb5kdc[31083](info): set up 2 sockets
Aug 11 00:20:04 dist-test-slave-f7th krb5kdc[31083](info): commencing operation
krb5kdc: starting...
W20260811 00:20:05.450039 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.007s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:20:05 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:20:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:20:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:20:05.756582 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:37681
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:37681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:06.136018 31100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:06.136353 31100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:06.136477 31100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:06.144181 31100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:20:06.144325 31100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:06.144372 31100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:06.144420 31100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:20:06.144466 31100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:20:06.155022 31100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:37681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:37681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:06.156421 31100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:06.157866 31100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:06.170965 31107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:06.171196 31105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:06.171010 31109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:06.171962 31100 server_base.cc:1034] running on GCE node
I20260811 00:20:06.172884 31100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:06.174584 31100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:06.175912 31100 hybrid_clock.cc:648] HybridClock initialized: now 1786407606175820 us; error 40 us; skew 500 ppm
I20260811 00:20:06.176327 31100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:06 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407606, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:06.179986 31100 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:20:06.181793 31100 webserver.cc:460] Webserver started at http://127.16.71.190:43569/ using document root <none> and password file <none>
I20260811 00:20:06.182585 31100 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:06.182714 31100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:06.183060 31100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:06.186616 31100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "1d6906ca5507497db2028b6acdfa4189"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "988021c727983c1b782acd06cd0a3f77"
server_key_iv: "1f8953dbbdc73f19b3f2c291a695ee5c"
server_key_version: "encryptionkey@0"
I20260811 00:20:06.187525 31100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "1d6906ca5507497db2028b6acdfa4189"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "988021c727983c1b782acd06cd0a3f77"
server_key_iv: "1f8953dbbdc73f19b3f2c291a695ee5c"
server_key_version: "encryptionkey@0"
I20260811 00:20:06.194008 31100 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:20:06.198226 31123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:06.199829 31100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:20:06.200045 31100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "1d6906ca5507497db2028b6acdfa4189"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "988021c727983c1b782acd06cd0a3f77"
server_key_iv: "1f8953dbbdc73f19b3f2c291a695ee5c"
server_key_version: "encryptionkey@0"
I20260811 00:20:06.200287 31100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:06.216362 31100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:06.221365 31100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:06.221693 31100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:06.243257 31100 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:37681
I20260811 00:20:06.243294 31222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:37681 every 8 connection(s)
I20260811 00:20:06.245087 31100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:20:06.249444 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 31100
I20260811 00:20:06.249718 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:20:06.250016 31224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:06.250577 16670 external_mini_cluster.cc:1428] Setting key b2aa0bed0db216315200e72ce720155d
I20260811 00:20:06.266391 31224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Bootstrap starting.
Aug 11 00:20:06 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:06.270347 31224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:06.271901 31224 log.cc:824] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:06.275274 31224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: No bootstrap required, opened a new log
I20260811 00:20:06.277264 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:06.253064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.16.71.190:37681)
0811 00:20:06.254076 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:06.254090 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:06.254145 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:20:06.255861 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:06.255891 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:06.256404 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:06.256861 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:06.258948 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:06.260371 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:06.262272 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:06.263180 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:06.263342 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:06.268791 (+ 5449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:06.268807 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:06.268836 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:06.268914 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:06.272812 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:06.273637 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:06.273646 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:06.273653 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:06.273840 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:06.274251 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:06.274255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:06.274258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:06.274867 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:06.275185 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:06.275666 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 00:20:06.276155 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":478,"threads_started":1}
I20260811 00:20:06.284273 31224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:06.284756 31224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:06.284866 31224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6906ca5507497db2028b6acdfa4189, State: Initialized, Role: FOLLOWER
I20260811 00:20:06.285610 31224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:06.285905 31224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:20:06.286054 31224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:20:06.286405 31224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:06.288926 31224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:06.289559 31224 leader_election.cc:302] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d6906ca5507497db2028b6acdfa4189; no voters:
I20260811 00:20:06.290205 31224 leader_election.cc:288] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:20:06.290359 31234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:06.291194 31234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 LEADER]: Becoming Leader. State: Replica: 1d6906ca5507497db2028b6acdfa4189, State: Running, Role: LEADER
I20260811 00:20:06.291751 31234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:06.292590 31224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:20:06.294829 31235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d6906ca5507497db2028b6acdfa4189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } } }
I20260811 00:20:06.296267 31235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:06.295102 31237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d6906ca5507497db2028b6acdfa4189. Latest consensus state: current_term: 1 leader_uuid: "1d6906ca5507497db2028b6acdfa4189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } } }
I20260811 00:20:06.297366 31237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:06.298414 31249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:20:06.303793 31249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:20:06.313627 31249 catalog_manager.cc:1347] Generated new cluster ID: a0514f4be7ce48c2a3bfa748ae7813bb
I20260811 00:20:06.313781 31249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:20:06.338836 31249 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:20:06.340685 31249 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:20:06.349220 31249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Generated new TSK 0
I20260811 00:20:06.350314 31249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:20:06.434564 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37681
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:06.836798 31266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:06.837157 31266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:06.837263 31266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:06.845791 31266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:06.845964 31266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:06.846096 31266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:20:06.857031 31266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:06.858527 31266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:06.860112 31266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:06.875891 31280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:06.876727 31284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:06.876711 31282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:06.877127 31266 server_base.cc:1034] running on GCE node
I20260811 00:20:06.877894 31266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:06.879060 31266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:06.880328 31266 hybrid_clock.cc:648] HybridClock initialized: now 1786407606880265 us; error 56 us; skew 500 ppm
I20260811 00:20:06.880903 31266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:06 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407606, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:06.884708 31266 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:20:06.886642 31266 webserver.cc:460] Webserver started at http://127.16.71.129:38879/ using document root <none> and password file <none>
I20260811 00:20:06.887465 31266 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:06.887590 31266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:06.887992 31266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:06.891383 31266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "f3f9b1e150a143bba602b738e4256755"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "a05b0096b5879902e33f1ad49db1e04c"
server_key_iv: "e9cb55e331e662a5bf4609e9a9221e71"
server_key_version: "encryptionkey@0"
I20260811 00:20:06.892298 31266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "f3f9b1e150a143bba602b738e4256755"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "a05b0096b5879902e33f1ad49db1e04c"
server_key_iv: "e9cb55e331e662a5bf4609e9a9221e71"
server_key_version: "encryptionkey@0"
I20260811 00:20:06.898628 31266 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 00:20:06.902895 31292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:06.904737 31266 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:20:06.904954 31266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "f3f9b1e150a143bba602b738e4256755"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "a05b0096b5879902e33f1ad49db1e04c"
server_key_iv: "e9cb55e331e662a5bf4609e9a9221e71"
server_key_version: "encryptionkey@0"
I20260811 00:20:06.905198 31266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:06.919755 31266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:06.924746 31266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:06.925071 31266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:06.926141 31266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:06.927748 31266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:06.927836 31266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:06.927964 31266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:06.928026 31266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:06.967044 31266 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:33669
I20260811 00:20:06.967092 31526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:33669 every 8 connection(s)
I20260811 00:20:06.968509 31266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:20:06.974494 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 31266
I20260811 00:20:06.974704 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:20:06.975363 16670 external_mini_cluster.cc:1428] Setting key 8a712abc9fadb328c91530feb79bca66
Aug 11 00:20:06 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407606, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:06.995152 31531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:06.972000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37509 (local address 127.16.71.190:37681)
0811 00:20:06.972561 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:06.972568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:06.973890 (+ 1322us) server_negotiation.cc:407] Connection header received
0811 00:20:06.976346 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:06.976352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:06.976460 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:06.976589 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:06.978475 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:06.979241 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:06.981234 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:06.981692 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:06.981837 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:06.986697 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:06.986706 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:06.986711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:06.986760 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:06.990302 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:06.991318 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:06.991325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:06.991328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:06.991505 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:06.992341 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:06.992348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:06.992352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:06.992820 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:06.993006 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:06.994464 (+ 1458us) server_negotiation.cc:299] Negotiation successful
0811 00:20:06.994766 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":271,"threads_started":1}
I20260811 00:20:06.996073 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:06.972506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:37509)
0811 00:20:06.973650 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:06.973723 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:06.975709 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:06.976937 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:06.976948 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:06.977575 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:06.978247 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:06.978269 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:06.979459 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:06.979464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:06.981022 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:06.981046 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:06.982001 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:06.982009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:06.982289 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:06.983083 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:06.983112 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:06.986280 (+ 3168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:06.990552 (+ 4272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:06.990563 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:06.990583 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:06.991136 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:06.991679 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:06.991684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:06.991686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:06.991965 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:06.993212 (+ 1247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:06.993225 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:06.993934 (+ 709us) client_negotiation.cc:769] Sending connection context
0811 00:20:06.994248 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 00:20:06.994779 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":433,"threads_started":1}
I20260811 00:20:06.998471 31529 heartbeater.cc:347] Connected to a master server at 127.16.71.190:37681
I20260811 00:20:06.998955 31529 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:07.000414 31529 heartbeater.cc:511] Master 127.16.71.190:37681 requested a full tablet report, sending...
I20260811 00:20:07.004326 31154 ts_manager.cc:194] Registered new tserver with Master: f3f9b1e150a143bba602b738e4256755 (127.16.71.129:33669)
I20260811 00:20:07.006119 31154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:37509
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:20:07.045770 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37681
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:07.417903 31536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:07.418257 31536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:07.418401 31536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:07.426081 31536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:07.426240 31536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:07.426391 31536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:20:07.436909 31536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:07.438447 31536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:07.440294 31536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:07.456244 31550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:07.456246 31547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:07.456518 31553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:07.457336 31536 server_base.cc:1034] running on GCE node
I20260811 00:20:07.458212 31536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:07.459403 31536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:07.460727 31536 hybrid_clock.cc:648] HybridClock initialized: now 1786407607460650 us; error 71 us; skew 500 ppm
I20260811 00:20:07.461166 31536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:07 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407607, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:07.469884 31536 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:20:07.471935 31536 webserver.cc:460] Webserver started at http://127.16.71.130:41953/ using document root <none> and password file <none>
I20260811 00:20:07.472812 31536 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:07.472970 31536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:07.473295 31536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:07.476605 31536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "7718ea09c4d84aec8c9190402d3f90de"
format_stamp: "Formatted at 2026-08-11 00:20:07 on dist-test-slave-f7th"
server_key: "b1ca6042d8cbdddf4ba461fde378b054"
server_key_iv: "29640a932ca8c2f87856e5bbfe8b4a22"
server_key_version: "encryptionkey@0"
I20260811 00:20:07.477581 31536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "7718ea09c4d84aec8c9190402d3f90de"
format_stamp: "Formatted at 2026-08-11 00:20:07 on dist-test-slave-f7th"
server_key: "b1ca6042d8cbdddf4ba461fde378b054"
server_key_iv: "29640a932ca8c2f87856e5bbfe8b4a22"
server_key_version: "encryptionkey@0"
I20260811 00:20:07.483559 31536 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:20:07.487721 31565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:07.489344 31536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:20:07.489550 31536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "7718ea09c4d84aec8c9190402d3f90de"
format_stamp: "Formatted at 2026-08-11 00:20:07 on dist-test-slave-f7th"
server_key: "b1ca6042d8cbdddf4ba461fde378b054"
server_key_iv: "29640a932ca8c2f87856e5bbfe8b4a22"
server_key_version: "encryptionkey@0"
I20260811 00:20:07.489782 31536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:07.507205 31536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:07.512215 31536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:07.512526 31536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:07.513693 31536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:07.515211 31536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:07.515303 31536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:07.515441 31536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:07.515506 31536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:07.552886 31536 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:34651
I20260811 00:20:07.552963 31770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:34651 every 8 connection(s)
I20260811 00:20:07.554369 31536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:20:07.556126 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 31536
I20260811 00:20:07.556324 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:20:07.556849 16670 external_mini_cluster.cc:1428] Setting key 9be04a68f2e1f7f5618e4bd7c9529a7e
Aug 11 00:20:07 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407607, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:07.583418 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:07.557630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:37977 (local address 127.16.71.190:37681)
0811 00:20:07.558197 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:07.558204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:07.560578 (+ 2374us) server_negotiation.cc:407] Connection header received
0811 00:20:07.563564 (+ 2986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:07.563569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:07.563663 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:07.564044 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:07.566539 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:07.567469 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:07.570281 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:07.570796 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:07.570925 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:07.576103 (+ 5178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:07.576109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:07.576113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:07.576158 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:07.579169 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:07.580424 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:07.580429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:07.580431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:07.580584 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:07.581215 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:07.581221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:07.581224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:07.581573 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:07.581712 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:07.582797 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 00:20:07.583111 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":277,"threads_started":1}
I20260811 00:20:07.584890 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:07.558329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:37977)
0811 00:20:07.560299 (+ 1970us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:07.560374 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:07.563057 (+ 2683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:07.564352 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:07.564367 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:07.565268 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:07.566238 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:07.566271 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:07.567718 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:07.567724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:07.569992 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:07.570034 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:07.571104 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:07.571112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:07.571460 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:07.572395 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:07.572432 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:07.575673 (+ 3241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:07.579435 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:07.579448 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:07.579472 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:07.580246 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:07.580730 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:07.580735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:07.580738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:07.581044 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:07.581739 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:07.581748 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:07.582292 (+ 544us) client_negotiation.cc:769] Sending connection context
0811 00:20:07.582645 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 00:20:07.583263 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1562,"thread_start_us":1297,"threads_started":1}
I20260811 00:20:07.586611 31771 heartbeater.cc:347] Connected to a master server at 127.16.71.190:37681
I20260811 00:20:07.586962 31771 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:07.587779 31771 heartbeater.cc:511] Master 127.16.71.190:37681 requested a full tablet report, sending...
I20260811 00:20:07.589402 31154 ts_manager.cc:194] Registered new tserver with Master: 7718ea09c4d84aec8c9190402d3f90de (127.16.71.130:34651)
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:20:07.590297 31154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:37977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:20:07.629390 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37681
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:08.005092 31779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:08.005437 31779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:08.005548 31779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 00:20:08.010191 31529 heartbeater.cc:503] Master 127.16.71.190:37681 was elected leader, sending a full tablet report...
W20260811 00:20:08.014400 31779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:08.014550 31779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:08.014735 31779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:20:08.026089 31779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:08.027632 31779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:08.029160 31779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:08.045130 31792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:08.045143 31797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:08.045172 31791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:08.045295 31779 server_base.cc:1034] running on GCE node
I20260811 00:20:08.046455 31779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:08.047588 31779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:08.048803 31779 hybrid_clock.cc:648] HybridClock initialized: now 1786407608048765 us; error 39 us; skew 500 ppm
I20260811 00:20:08.049178 31779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:08 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407608, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:08.052663 31779 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:20:08.054397 31779 webserver.cc:460] Webserver started at http://127.16.71.131:40045/ using document root <none> and password file <none>
I20260811 00:20:08.055164 31779 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:08.055292 31779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:08.055649 31779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:08.059044 31779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "3e8f826b64a9456c907575b60a36b11e"
format_stamp: "Formatted at 2026-08-11 00:20:08 on dist-test-slave-f7th"
server_key: "bfab9e0c7e90c1858ac0bfa99af3135b"
server_key_iv: "c73f672d65b03a8e80aa98a4f9774b2c"
server_key_version: "encryptionkey@0"
I20260811 00:20:08.059940 31779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "3e8f826b64a9456c907575b60a36b11e"
format_stamp: "Formatted at 2026-08-11 00:20:08 on dist-test-slave-f7th"
server_key: "bfab9e0c7e90c1858ac0bfa99af3135b"
server_key_iv: "c73f672d65b03a8e80aa98a4f9774b2c"
server_key_version: "encryptionkey@0"
I20260811 00:20:08.066118 31779 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260811 00:20:08.070465 31804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:08.072306 31779 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:20:08.072536 31779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "3e8f826b64a9456c907575b60a36b11e"
format_stamp: "Formatted at 2026-08-11 00:20:08 on dist-test-slave-f7th"
server_key: "bfab9e0c7e90c1858ac0bfa99af3135b"
server_key_iv: "c73f672d65b03a8e80aa98a4f9774b2c"
server_key_version: "encryptionkey@0"
I20260811 00:20:08.072810 31779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:08.097254 31779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:08.102411 31779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:08.102746 31779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:08.103857 31779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:08.105547 31779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:08.105635 31779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:08.105742 31779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:08.105791 31779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:08.138952 31779 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:36939
I20260811 00:20:08.139020 32006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:36939 every 8 connection(s)
I20260811 00:20:08.140357 31779 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:20:08.141034 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 31779
I20260811 00:20:08.141237 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:20:08.141817 16670 external_mini_cluster.cc:1428] Setting key 9581b42654baebafa0ea9583b0d93971
Aug 11 00:20:08 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407608, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:08.162096 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:08.143188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:55597 (local address 127.16.71.190:37681)
0811 00:20:08.143717 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:08.143722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:08.145183 (+ 1461us) server_negotiation.cc:407] Connection header received
0811 00:20:08.146899 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:08.146905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:08.147005 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:08.147108 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:08.148772 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.149548 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.151190 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.151614 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.151734 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:08.155779 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:08.155785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:08.155788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:08.155826 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:08.158606 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.159416 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.159420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.159423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.159553 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.160009 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.160019 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.160022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.160309 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:08.160457 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:08.161410 (+ 953us) server_negotiation.cc:299] Negotiation successful
0811 00:20:08.161681 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":259,"threads_started":1}
I20260811 00:20:08.163110 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:08.143872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:55597)
0811 00:20:08.144751 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:08.144800 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:08.146578 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:08.147386 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:08.147395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:08.147942 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:08.148557 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:08.148582 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.149707 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.149711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:08.151022 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:08.151038 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.151867 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.151876 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:08.152153 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:08.153070 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:08.153097 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:08.155485 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:08.158769 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:08.158780 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:08.158797 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:08.159278 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:08.159666 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:08.159671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:08.159675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:08.159888 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:08.160458 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:08.160467 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:08.160990 (+ 523us) client_negotiation.cc:769] Sending connection context
0811 00:20:08.161267 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 00:20:08.161883 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":324,"threads_started":1}
I20260811 00:20:08.164680 32007 heartbeater.cc:347] Connected to a master server at 127.16.71.190:37681
I20260811 00:20:08.165011 32007 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:08.165782 32007 heartbeater.cc:511] Master 127.16.71.190:37681 requested a full tablet report, sending...
I20260811 00:20:08.167383 31154 ts_manager.cc:194] Registered new tserver with Master: 3e8f826b64a9456c907575b60a36b11e (127.16.71.131:36939)
I20260811 00:20:08.168327 31154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:55597
I20260811 00:20:08.170325 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab.
Entry for principal oryx/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab.
WARNING: no policy specified for oryx/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab.
Entry for principal oryx/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab.
WARNING: no policy specified for oryx/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab.
Entry for principal oryx/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab.
I20260811 00:20:08.360385 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 31266
Aug 11 00:20:08 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:20:08.379465 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:08.361687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.16.71.129:33669)
0811 00:20:08.362383 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:08.362393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:08.362887 (+ 494us) server_negotiation.cc:407] Connection header received
0811 00:20:08.363078 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:08.363087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:08.363318 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:08.363464 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:08.365011 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.366065 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.367377 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.367963 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.368084 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:08.372733 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:08.372751 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:08.372770 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:08.372839 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:08.376125 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.377124 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.377132 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.377135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.377294 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.377728 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.377732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.377734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.378170 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:08.378472 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:08.378820 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 00:20:08.379121 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":345,"threads_started":1}
I20260811 00:20:08.380836 31518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:08.467495 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 31266
I20260811 00:20:08.495772 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 31536
Aug 11 00:20:08 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:20:08.514376 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:08.496814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49076 (local address 127.16.71.130:34651)
0811 00:20:08.497429 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:08.497437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:08.497466 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:20:08.497545 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:08.497552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:08.497773 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:08.497926 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:08.499562 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.500809 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.502244 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.502901 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.503020 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:08.508044 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:08.508063 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:08.508081 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:08.508128 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:08.511312 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.512199 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.512208 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.512211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.512366 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.512832 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.512836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.512839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.513289 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:08.513566 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:08.513801 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 00:20:08.514071 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":320,"threads_started":1}
I20260811 00:20:08.515756 31765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:08.601473 31771 heartbeater.cc:503] Master 127.16.71.190:37681 was elected leader, sending a full tablet report...
I20260811 00:20:08.603070 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 31536
W20260811 00:20:08.627234 31127 connection.cc:535] server connection from 127.16.71.130:37977 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:20:08.627626 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 31779
Aug 11 00:20:08 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:20:08.649052 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:08.629088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.16.71.131:36939)
0811 00:20:08.629702 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:08.629710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:08.629745 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:20:08.629815 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:08.629823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:08.630058 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:08.630271 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:08.631826 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.633116 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.634678 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:08.635338 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:08.635483 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:08.641258 (+ 5775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:08.641277 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:08.641296 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:08.641362 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:08.645109 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.646407 (+ 1298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.646418 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.646421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.646604 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:08.647098 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:08.647102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:08.647105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:08.647581 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:08.647910 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:08.648314 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 00:20:08.648682 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":289,"threads_started":1}
I20260811 00:20:08.650554 31996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:08.737195 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 31779
I20260811 00:20:08.762490 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 31100
I20260811 00:20:08.763823 31216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:08.870875 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 31100
I20260811 00:20:08.897536 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:37681
--webserver_interface=127.16.71.190
--webserver_port=43569
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:37681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:09.271340 32035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:09.271720 32035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:09.271853 32035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:09.279537 32035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:20:09.279700 32035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:09.279784 32035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:09.279863 32035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:20:09.279927 32035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:20:09.290627 32035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:37681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:37681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=43569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:09.292099 32035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:09.293479 32035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:09.305997 32046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:09.305959 32048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:09.305927 32045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:09.306715 32035 server_base.cc:1034] running on GCE node
I20260811 00:20:09.307629 32035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:09.309281 32035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:09.310554 32035 hybrid_clock.cc:648] HybridClock initialized: now 1786407609310498 us; error 43 us; skew 500 ppm
I20260811 00:20:09.311002 32035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:09 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:09.314442 32035 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:20:09.316159 32035 webserver.cc:460] Webserver started at http://127.16.71.190:43569/ using document root <none> and password file <none>
I20260811 00:20:09.316957 32035 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:09.317122 32035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:09.323226 32035 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 00:20:09.326574 32062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:09.328452 32035 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 00:20:09.328730 32035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "1d6906ca5507497db2028b6acdfa4189"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "988021c727983c1b782acd06cd0a3f77"
server_key_iv: "1f8953dbbdc73f19b3f2c291a695ee5c"
server_key_version: "encryptionkey@0"
I20260811 00:20:09.329871 32035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:09.347851 32035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:09.353400 32035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:09.353706 32035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:09.374203 32035 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:37681
I20260811 00:20:09.374240 32156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:37681 every 8 connection(s)
I20260811 00:20:09.376003 32035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:20:09.379217 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 32035
I20260811 00:20:09.380245 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:33669
--local_ip_for_outbound_sockets=127.16.71.129
--tserver_master_addrs=127.16.71.190:37681
--webserver_port=38879
--webserver_interface=127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:20:09.386023 32157 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:20:09.391258 32157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Bootstrap starting.
I20260811 00:20:09.414734 32157 log.cc:824] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:09.425805 32157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:20:09.426430 32157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Bootstrap complete.
I20260811 00:20:09.436718 32157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:09.437750 32157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6906ca5507497db2028b6acdfa4189, State: Initialized, Role: FOLLOWER
I20260811 00:20:09.438448 32157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:09.438722 32157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:20:09.438849 32157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:20:09.439038 32157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:20:09.443478 32157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:09.444162 32157 leader_election.cc:302] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d6906ca5507497db2028b6acdfa4189; no voters:
I20260811 00:20:09.444890 32157 leader_election.cc:288] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:20:09.445115 32168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:20:09.446439 32168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [term 2 LEADER]: Becoming Leader. State: Replica: 1d6906ca5507497db2028b6acdfa4189, State: Running, Role: LEADER
I20260811 00:20:09.447270 32168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } }
I20260811 00:20:09.447517 32157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:20:09.450623 32173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d6906ca5507497db2028b6acdfa4189. Latest consensus state: current_term: 2 leader_uuid: "1d6906ca5507497db2028b6acdfa4189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } } }
I20260811 00:20:09.451078 32173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:09.451618 32169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d6906ca5507497db2028b6acdfa4189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6906ca5507497db2028b6acdfa4189" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 37681 } } }
I20260811 00:20:09.451921 32169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:09.459132 32181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:20:09.466557 32181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:20:09.468580 32181 catalog_manager.cc:1259] Loaded cluster ID: a0514f4be7ce48c2a3bfa748ae7813bb
I20260811 00:20:09.468755 32181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:20:09.473171 32181 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:20:09.475066 32181 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d6906ca5507497db2028b6acdfa4189: Loaded TSK: 0
I20260811 00:20:09.476912 32181 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:20:09.775672 32160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:09.776028 32160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:09.776165 32160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:09.784248 32160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:09.784407 32160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:09.784576 32160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:20:09.795670 32160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.129.keytab
--principal=oryx/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:33669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=38879
--tserver_master_addrs=127.16.71.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:09.797266 32160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:09.798733 32160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:09.815331 32230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:09.815274 32221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:09.815526 32218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:09.816737 32160 server_base.cc:1034] running on GCE node
I20260811 00:20:09.817548 32160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:09.818747 32160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:09.820039 32160 hybrid_clock.cc:648] HybridClock initialized: now 1786407609819930 us; error 75 us; skew 500 ppm
I20260811 00:20:09.820432 32160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:09 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407609, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:09.824331 32160 init.cc:377] Logged in from keytab as oryx/127.16.71.129@KRBTEST.COM (short username oryx)
I20260811 00:20:09.826190 32160 webserver.cc:460] Webserver started at http://127.16.71.129:38879/ using document root <none> and password file <none>
I20260811 00:20:09.827018 32160 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:09.827179 32160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:09.833977 32160 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:20:09.837539 32241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:09.839251 32160 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:20:09.839469 32160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "f3f9b1e150a143bba602b738e4256755"
format_stamp: "Formatted at 2026-08-11 00:20:06 on dist-test-slave-f7th"
server_key: "a05b0096b5879902e33f1ad49db1e04c"
server_key_iv: "e9cb55e331e662a5bf4609e9a9221e71"
server_key_version: "encryptionkey@0"
I20260811 00:20:09.840579 32160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:09.862056 32160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:09.867144 32160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:09.867449 32160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:09.868585 32160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:09.870182 32160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:09.870285 32160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:09.870397 32160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:09.870465 32160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:09.905402 32160 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:33669
I20260811 00:20:09.905498 32450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:33669 every 8 connection(s)
I20260811 00:20:09.907063 32160 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:20:09.915511 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 32160
I20260811 00:20:09.916123 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:34651
--local_ip_for_outbound_sockets=127.16.71.130
--tserver_master_addrs=127.16.71.190:37681
--webserver_port=41953
--webserver_interface=127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:20:09 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:09 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:20:10.341439 32457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:10.341742 32457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:10.341863 32457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:10.349818 32457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:10.349956 32457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:10.350106 32457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:20:10.360723 32457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.130.keytab
--principal=oryx/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:34651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=41953
--tserver_master_addrs=127.16.71.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:10.362239 32457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:10.363668 32457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:10.380643 32470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:10.380980 32472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:10.381511 32468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:10.382730 32457 server_base.cc:1034] running on GCE node
I20260811 00:20:10.383550 32457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:10.384959 32457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:10.386251 32457 hybrid_clock.cc:648] HybridClock initialized: now 1786407610386187 us; error 70 us; skew 500 ppm
I20260811 00:20:10.386610 32457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:10 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407610, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:10.390393 32457 init.cc:377] Logged in from keytab as oryx/127.16.71.130@KRBTEST.COM (short username oryx)
I20260811 00:20:10.392163 32457 webserver.cc:460] Webserver started at http://127.16.71.130:41953/ using document root <none> and password file <none>
I20260811 00:20:10.393254 32457 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:10.393462 32457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:10.400357 32457 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:20:10.403906 32481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:10.405860 32457 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:20:10.406077 32457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "7718ea09c4d84aec8c9190402d3f90de"
format_stamp: "Formatted at 2026-08-11 00:20:07 on dist-test-slave-f7th"
server_key: "b1ca6042d8cbdddf4ba461fde378b054"
server_key_iv: "29640a932ca8c2f87856e5bbfe8b4a22"
server_key_version: "encryptionkey@0"
I20260811 00:20:10.407158 32457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:10.421972 32457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:10.427093 32457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:10.427551 32457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:10.428925 32457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:10.430912 32457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:10.431027 32457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:10.431124 32457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:10.431200 32457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:10.472767 32457 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:34651
I20260811 00:20:10.472826 32682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:34651 every 8 connection(s)
I20260811 00:20:10.474457 32457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:20:10.477418 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 32457
I20260811 00:20:10.478305 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:36939
--local_ip_for_outbound_sockets=127.16.71.131
--tserver_master_addrs=127.16.71.190:37681
--webserver_port=40045
--webserver_interface=127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:20:10 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:10 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:20:11.009001 32456 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.087s user 0.006s sys 0.009s
I20260811 00:20:11.011787 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:09.910459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:36099)
0811 00:20:09.911468 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:09.911524 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:09.913483 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:09.915199 (+ 1716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:09.915212 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:09.915929 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:09.916739 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:09.916767 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:09.918810 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:09.918815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:09.920108 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:09.920125 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:09.921325 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:09.921334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:09.921644 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:09.922328 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:09.922362 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:11.010146 (+1087784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":381,"threads_started":1}
W20260811 00:20:11.012079 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:09.910440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:36099 (local address 127.16.71.190:37681)
0811 00:20:09.911374 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:09.911392 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:09.911659 (+ 267us) server_negotiation.cc:407] Connection header received
0811 00:20:09.914178 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:09.914194 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:09.914603 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:09.914979 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:09.916984 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:09.918574 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:09.920290 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:09.921053 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:09.921258 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.010938 (+1089680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:36099: BlockingRecv error: recv got EOF from 127.16.71.129:36099 (error 108)
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":371,"threads_started":1}
W20260811 00:20:11.014887 32451 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:37681 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.71.190:37681: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:11.027333 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:11.016602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:59701)
0811 00:20:11.017283 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:11.017307 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:11.017457 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:11.017973 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:11.017976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:11.018367 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:11.018546 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.018555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.020039 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.020045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.021214 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.021233 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.022036 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.022043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.022205 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.022842 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:11.022863 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:11.027006 (+ 4143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204}
W20260811 00:20:11.027570 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:11.016854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:59701 (local address 127.16.71.190:37681)
0811 00:20:11.017415 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:11.017421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:11.017442 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:20:11.017625 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:11.017632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:11.017706 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:11.017764 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:11.018725 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.019817 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.021402 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.021825 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.021947 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.027311 (+ 5364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:59701: BlockingRecv error: recv got EOF from 127.16.71.129:59701 (error 108)
Metrics: {"server-negotiator.queue_time_us":430}
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:11.038576 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:11.028750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:59931)
0811 00:20:11.029107 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:11.029127 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:11.029266 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:11.029617 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:11.029622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:11.030022 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:11.030206 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.030217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.031327 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.031333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.032355 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.032380 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.033291 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.033297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.033514 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.034201 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:11.034226 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:11.038227 (+ 4001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245}
W20260811 00:20:11.038765 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:11.028911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:59931 (local address 127.16.71.190:37681)
0811 00:20:11.029111 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:11.029117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:11.029200 (+ 83us) server_negotiation.cc:407] Connection header received
0811 00:20:11.029393 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:11.029398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:11.029467 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:11.029551 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:11.030369 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.031150 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.032537 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.033113 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.033240 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.038533 (+ 5293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:59931: BlockingRecv error: recv got EOF from 127.16.71.129:59931 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 00:20:11.039211 32451 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:20:11.134588 32690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:11.135138 32690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:11.135335 32690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:11.150103 32690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:11.150339 32690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:11.150571 32690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:20:11.164420 32690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/krb5kdc/oryx_127.16.71.131.keytab
--principal=oryx/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:36939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=40045
--tserver_master_addrs=127.16.71.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:11.166059 32690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:11.167551 32690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:11.185825 32704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:11.187031 32707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:11.188931 32712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:11.188971 32690 server_base.cc:1034] running on GCE node
I20260811 00:20:11.190130 32690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:11.191804 32690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:11.193145 32690 hybrid_clock.cc:648] HybridClock initialized: now 1786407611193039 us; error 54 us; skew 500 ppm
I20260811 00:20:11.193727 32690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407611, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:11.199400 32690 init.cc:377] Logged in from keytab as oryx/127.16.71.131@KRBTEST.COM (short username oryx)
I20260811 00:20:11.202310 32690 webserver.cc:460] Webserver started at http://127.16.71.131:40045/ using document root <none> and password file <none>
I20260811 00:20:11.203735 32690 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:11.204116 32690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:11.214358 32690 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:20:11.219058 32721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:11.221354 32690 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 00:20:11.221648 32690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "3e8f826b64a9456c907575b60a36b11e"
format_stamp: "Formatted at 2026-08-11 00:20:08 on dist-test-slave-f7th"
server_key: "bfab9e0c7e90c1858ac0bfa99af3135b"
server_key_iv: "c73f672d65b03a8e80aa98a4f9774b2c"
server_key_version: "encryptionkey@0"
I20260811 00:20:11.223497 32690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:11.274991 32690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:11.282390 32690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:11.282792 32690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:11.284338 32690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:11.287101 32690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:11.287285 32690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:11.287431 32690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:11.287529 32690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:11.326481 32690 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:36939
I20260811 00:20:11.326509 460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:36939 every 8 connection(s)
I20260811 00:20:11.329037 32690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:20:11.338562 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 32690
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:11.372954 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:11.339622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.16.71.190:37681)
0811 00:20:11.355977 (+ 16355us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:11.355984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:11.356005 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:20:11.356087 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:11.356093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:11.356164 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:11.356265 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:11.358033 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.358935 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.360935 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.361603 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.361756 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.364979 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:11.364992 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:11.365031 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:11.365088 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:11.368722 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:11.369693 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:11.369703 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:11.369711 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:11.369955 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:11.370413 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:11.370418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:11.370420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:11.371066 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:11.371436 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:11.371997 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 00:20:11.372471 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16183,"thread_start_us":16318,"threads_started":1}
W20260811 00:20:11.656893 32689 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.153s user 0.003s sys 0.012s
W20260811 00:20:11.658684 32688 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:10.477635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:40963 (local address 127.16.71.190:37681)
0811 00:20:10.494233 (+ 16598us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:10.494241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:10.494994 (+ 753us) server_negotiation.cc:407] Connection header received
0811 00:20:10.497236 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:10.497243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:10.497361 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:10.497521 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:10.499552 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:10.500337 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:10.502199 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:10.502770 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:10.502900 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.658253 (+1155353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:40963: BlockingRecv error: recv got EOF from 127.16.71.130:40963 (error 108)
Metrics: {"server-negotiator.queue_time_us":16409,"thread_start_us":16255,"threads_started":1}
I20260811 00:20:11.659554 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:10.478426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:40963)
0811 00:20:10.494760 (+ 16334us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:10.494821 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:10.496838 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:10.497859 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:10.497874 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:10.498661 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:10.499304 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:10.499325 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:10.500560 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:10.500564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:10.501995 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:10.502013 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:10.503105 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:10.503113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:10.503396 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:10.504116 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:10.504141 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:11.658049 (+1153908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15960,"thread_start_us":15685,"threads_started":1}
W20260811 00:20:11.661106 32684 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:37681 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.71.190:37681: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:11.673312 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:11.662032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:45525)
0811 00:20:11.662232 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:11.662249 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:11.662354 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:11.662784 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:11.662789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:11.663203 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:11.663402 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.663413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.665218 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.665224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.666291 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.666308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.667538 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.667544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.667689 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.668518 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:11.668544 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:11.673074 (+ 4530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 00:20:11.674108 32688 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:11.662139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:45525 (local address 127.16.71.190:37681)
0811 00:20:11.662353 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:11.662360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:11.662388 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:20:11.662467 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:11.662473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:11.662580 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:11.662711 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:11.663574 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.664969 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.666472 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.667263 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.667433 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.673399 (+ 5966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:45525: BlockingRecv error: recv got EOF from 127.16.71.130:45525 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:11 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:11.686038 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:11.674129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:52297)
0811 00:20:11.674624 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:11.674645 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:11.674779 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:11.675181 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:11.675185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:11.675573 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:11.675749 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.675758 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.677299 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.677304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.678309 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.678324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.679390 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.679395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.679722 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.680638 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:11.680668 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:11.685804 (+ 5136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371}
W20260811 00:20:11.686412 32684 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:20:11.686930 470 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:11.674373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:52297 (local address 127.16.71.190:37681)
0811 00:20:11.674654 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:11.674660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:11.674738 (+ 78us) server_negotiation.cc:407] Connection header received
0811 00:20:11.674897 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:11.674904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:11.675002 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:11.675124 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:11.675887 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.677139 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.678451 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.679184 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.679335 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.686559 (+ 7224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:52297: BlockingRecv error: recv got EOF from 127.16.71.130:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:12.052373 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:12.040058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:43753)
0811 00:20:12.040742 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:12.040762 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:12.040902 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:12.041364 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:12.041368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:12.041845 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:12.042054 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.042070 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.043789 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.043795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.045071 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.045096 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.046280 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.046285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.046454 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.047201 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:12.047221 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:12.051933 (+ 4712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":375,"threads_started":1}
W20260811 00:20:12.052407 470 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:12.040165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:43753 (local address 127.16.71.190:37681)
0811 00:20:12.040420 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:12.040429 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:12.040859 (+ 430us) server_negotiation.cc:407] Connection header received
0811 00:20:12.041027 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:12.041035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:12.041139 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:12.041280 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:12.042237 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.043599 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.045250 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.046070 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.046224 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.052181 (+ 5957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:43753: BlockingRecv error: recv got EOF from 127.16.71.129:43753 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 00:20:12.400643 463 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.054s user 0.000s sys 0.025s
W20260811 00:20:12.402176 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:11.332302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:52483 (local address 127.16.71.190:37681)
0811 00:20:11.332570 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:11.332579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:11.333910 (+ 1331us) server_negotiation.cc:407] Connection header received
0811 00:20:11.336246 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:11.336255 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:11.336377 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:11.336522 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:11.339052 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.340355 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.343554 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.344444 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.344569 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.401876 (+1057307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:52483: BlockingRecv error: recv got EOF from 127.16.71.131:52483 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:20:12.402725 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:11.332843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:52483)
0811 00:20:11.333726 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:11.333788 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:11.335878 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:11.336887 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:11.336897 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:11.337589 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:11.338703 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.338737 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.340708 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.340716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.343236 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:11.343263 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:11.345087 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:11.345096 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:11.345520 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:11.346552 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:11.346591 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:12.401563 (+1054972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":333,"threads_started":1}
W20260811 00:20:12.403754 461 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:37681 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.71.190:37681: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:12.414157 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:12.404510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:38815)
0811 00:20:12.404737 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:12.404762 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:12.404911 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:12.405281 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:12.405286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:12.405697 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:12.405877 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.405887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.407024 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.407028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.408032 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.408047 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.408916 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.408923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.409077 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.409836 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:12.409857 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:12.413909 (+ 4052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 00:20:12.414372 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:12.404639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:38815 (local address 127.16.71.190:37681)
0811 00:20:12.404909 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:12.404914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:12.404935 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:20:12.405067 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:12.405071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:12.405137 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:12.405221 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:12.406042 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.406862 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.408200 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.408714 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.408835 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.414153 (+ 5318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:38815: BlockingRecv error: recv got EOF from 127.16.71.131:38815 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:12.423877 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:12.414918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:58825)
0811 00:20:12.415139 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:12.415159 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:12.415283 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:12.415614 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:12.415620 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:12.416010 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:12.416191 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.416201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.417295 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.417301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.418311 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.418326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.419107 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.419113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.419324 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.419988 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:12.420007 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:12.423595 (+ 3588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:20:12.424217 461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:20:12.424386 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:12.415050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:58825 (local address 127.16.71.190:37681)
0811 00:20:12.415235 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:12.415240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:12.415258 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:20:12.415402 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:12.415408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:12.415469 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:12.415558 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:12.416343 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.417145 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.418475 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.418938 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.419055 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.423877 (+ 4822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:58825: BlockingRecv error: recv got EOF from 127.16.71.131:58825 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:12 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:12.702179 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:12.687208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:45513)
0811 00:20:12.687807 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:12.687823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:12.688031 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:12.688416 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:12.688421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:12.688968 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:12.689128 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.689139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.690217 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.690221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.691242 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:12.691255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.692050 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.692057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:12.692217 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.697327 (+ 5110us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:12.697374 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:12.701819 (+ 4445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":349,"threads_started":1}
W20260811 00:20:12.702431 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:12.687339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:45513 (local address 127.16.71.190:37681)
0811 00:20:12.687590 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:12.687597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:12.687918 (+ 321us) server_negotiation.cc:407] Connection header received
0811 00:20:12.688165 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:12.688169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:12.688263 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:12.688350 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:12.689291 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.690056 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.691408 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:12.691841 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:12.691968 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:12.702172 (+ 10204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:45513: BlockingRecv error: recv got EOF from 127.16.71.130:45513 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 11 00:20:13 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:13 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:13.063573 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:13.053706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:58729)
0811 00:20:13.054257 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:13.054274 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:13.054409 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:13.054782 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:13.054785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:13.055235 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:13.055435 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:13.055444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.056596 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.056601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:13.057677 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:13.057696 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.058697 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.058703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:13.058895 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:13.059524 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:13.059543 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:13.063300 (+ 3757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":289,"threads_started":1}
W20260811 00:20:13.063831 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:13.053830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:58729 (local address 127.16.71.190:37681)
0811 00:20:13.054037 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:13.054041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:13.054355 (+ 314us) server_negotiation.cc:407] Connection header received
0811 00:20:13.054569 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:13.054574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:13.054649 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:13.054749 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:13.055600 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.056392 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.057876 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.058517 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.058646 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:13.063564 (+ 4918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:58729: BlockingRecv error: recv got EOF from 127.16.71.129:58729 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 00:20:13 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:13 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:13.434792 475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:13.424983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:44603)
0811 00:20:13.425718 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:13.425734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:13.425889 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:13.426291 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:13.426295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:13.426736 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:13.426912 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:13.426922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.428003 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.428007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:13.429038 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:13.429053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.429848 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.429856 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:13.430025 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:13.430671 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:13.430690 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:13.434463 (+ 3773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":522,"threads_started":1}
W20260811 00:20:13.434993 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:13.425178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:44603 (local address 127.16.71.190:37681)
0811 00:20:13.425358 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:13.425364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:13.425823 (+ 459us) server_negotiation.cc:407] Connection header received
0811 00:20:13.426082 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:13.426089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:13.426156 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:13.426237 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:13.427078 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.427841 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.429195 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.429633 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.429753 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:13.434737 (+ 4984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:44603: BlockingRecv error: recv got EOF from 127.16.71.131:44603 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 00:20:13 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:13 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:13.713855 478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:13.703486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:56677)
0811 00:20:13.704221 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:13.704241 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:13.704405 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:13.704844 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:13.704850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:13.705405 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:13.705618 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:13.705629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.706866 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.706871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:13.707872 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:13.707888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.708726 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.708731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:13.708913 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:13.709587 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:13.709605 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:13.713584 (+ 3979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":288,"threads_started":1}
W20260811 00:20:13.714226 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:13.703767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:56677 (local address 127.16.71.190:37681)
0811 00:20:13.703971 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:13.703976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:13.704335 (+ 359us) server_negotiation.cc:407] Connection header received
0811 00:20:13.704535 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:13.704541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:13.704670 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:13.704764 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:13.705818 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.706632 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.708052 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:13.708514 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:13.708654 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:13.713980 (+ 5326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:56677: BlockingRecv error: recv got EOF from 127.16.71.130:56677 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 00:20:14 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:14 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:14.074416 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:14.064866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:59721)
0811 00:20:14.065524 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:14.065545 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:14.065694 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:14.066034 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:14.066039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:14.066470 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:14.066655 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:14.066665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.067731 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.067737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:14.068744 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:14.068759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.069479 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.069485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:14.069678 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:14.070285 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:14.070304 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:14.074165 (+ 3861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":379,"threads_started":1}
W20260811 00:20:14.074667 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:14.065051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:59721 (local address 127.16.71.190:37681)
0811 00:20:14.065265 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:14.065271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:14.065618 (+ 347us) server_negotiation.cc:407] Connection header received
0811 00:20:14.065817 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:14.065823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:14.065891 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:14.065974 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:14.066818 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.067584 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.068894 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.069325 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.069449 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:14.074430 (+ 4981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:59721: BlockingRecv error: recv got EOF from 127.16.71.129:59721 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 00:20:14 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:14 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:14.446996 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:14.436064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:56565)
0811 00:20:14.436642 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:14.436661 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:14.436795 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:14.437214 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:14.437219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:14.437670 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:14.437900 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:14.437910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.439122 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.439127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:14.440120 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:14.440137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.441130 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.441136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:14.441282 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:14.442155 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:14.442176 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:14.446702 (+ 4526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":317,"threads_started":1}
W20260811 00:20:14.447157 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:14.436146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:56565 (local address 127.16.71.190:37681)
0811 00:20:14.436386 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:14.436395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:14.436747 (+ 352us) server_negotiation.cc:407] Connection header received
0811 00:20:14.436948 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:14.436952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:14.437030 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:14.437125 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:14.438121 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.438883 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.440343 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.440931 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.441053 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:14.446929 (+ 5876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:56565: BlockingRecv error: recv got EOF from 127.16.71.131:56565 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 00:20:14 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:14 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:14.724836 482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:14.715053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:53165)
0811 00:20:14.715724 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:14.715746 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:14.715972 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:14.716397 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:14.716402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:14.716933 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:14.717121 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:14.717132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.718202 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.718208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:14.719144 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:14.719160 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.719919 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.719925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:14.720107 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:14.720757 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:14.720776 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:14.724513 (+ 3737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"spinlock_wait_cycles":35200,"thread_start_us":379,"threads_started":1}
W20260811 00:20:14.725142 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:14.715205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:53165 (local address 127.16.71.190:37681)
0811 00:20:14.715454 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:14.715459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:14.715855 (+ 396us) server_negotiation.cc:407] Connection header received
0811 00:20:14.716134 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:14.716140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:14.716231 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:14.716331 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:14.717281 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.718055 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.719308 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:14.719752 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:14.719866 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:14.724883 (+ 5017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:53165: BlockingRecv error: recv got EOF from 127.16.71.130:53165 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 00:20:15 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:15 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:15.085565 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:15.075579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:37251)
0811 00:20:15.076268 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:15.076285 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:15.076436 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:15.076873 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:15.076878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:15.077293 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:15.077478 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:15.077489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.078499 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.078506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:15.079545 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:15.079560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.080290 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.080297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:15.080485 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:15.081163 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:15.081181 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:15.085192 (+ 4011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":325,"threads_started":1}
W20260811 00:20:15.085759 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:15.075598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37251 (local address 127.16.71.190:37681)
0811 00:20:15.075802 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:15.075809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:15.076372 (+ 563us) server_negotiation.cc:407] Connection header received
0811 00:20:15.076575 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:15.076581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:15.076714 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:15.076803 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:15.077636 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.078359 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.079708 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.080137 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.080252 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:15.085512 (+ 5260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:37251: BlockingRecv error: recv got EOF from 127.16.71.129:37251 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 00:20:15 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:15 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:15.458036 487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:15.448192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:47913)
0811 00:20:15.449164 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:15.449187 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:15.449340 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:15.449687 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:15.449691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:15.450161 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:15.450333 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:15.450343 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.451382 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.451387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:15.452318 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:15.452332 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.453203 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.453210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:15.453397 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:15.454032 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:15.454052 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:15.457615 (+ 3563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":415,"threads_started":1}
W20260811 00:20:15.458149 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:15.448321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:47913 (local address 127.16.71.190:37681)
0811 00:20:15.448573 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:15.448580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:15.449275 (+ 695us) server_negotiation.cc:407] Connection header received
0811 00:20:15.449466 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:15.449473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:15.449557 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:15.449653 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:15.450484 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.451223 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.452467 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.453053 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.453168 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:15.457900 (+ 4732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:47913: BlockingRecv error: recv got EOF from 127.16.71.131:47913 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 11 00:20:15 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:15 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:15.736348 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:15.726154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:35609)
0811 00:20:15.726761 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:15.726779 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:15.726947 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:15.727316 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:15.727320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:15.727736 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:15.727963 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:15.727974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.729093 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.729099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:15.730145 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:15.730164 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.731003 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.731010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:15.731241 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:15.731979 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:15.731998 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:15.736063 (+ 4065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":316,"threads_started":1}
W20260811 00:20:15.736678 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:15.726389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:35609 (local address 127.16.71.190:37681)
0811 00:20:15.726673 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:15.726681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:15.726887 (+ 206us) server_negotiation.cc:407] Connection header received
0811 00:20:15.727085 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:15.727090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:15.727157 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:15.727244 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:15.728127 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.728934 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.730309 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:15.730808 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:15.730943 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:15.736361 (+ 5418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:35609: BlockingRecv error: recv got EOF from 127.16.71.130:35609 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 00:20:16 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:16 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:16.096729 489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:16.086758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:38687)
0811 00:20:16.087352 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:16.087368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:16.087533 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:16.087853 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:16.087857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:16.088338 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:16.088507 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:16.088518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.089650 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.089655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:16.090618 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:16.090634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.091441 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.091448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:16.091646 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:16.092343 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:16.092368 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:16.096423 (+ 4055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":330,"threads_started":1}
W20260811 00:20:16.097031 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:16.086911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:38687 (local address 127.16.71.190:37681)
0811 00:20:16.087149 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:16.087157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:16.087454 (+ 297us) server_negotiation.cc:407] Connection header received
0811 00:20:16.087659 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:16.087664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:16.087727 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:16.087811 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:16.088697 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.089498 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.090788 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.091277 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.091398 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:16.096719 (+ 5321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:38687: BlockingRecv error: recv got EOF from 127.16.71.129:38687 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 00:20:16 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:16 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:16.469873 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:16.459280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:42785)
0811 00:20:16.459843 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:16.459867 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:16.460033 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:16.460588 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:16.460592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:16.461027 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:16.461214 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:16.461226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.462348 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.462360 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:16.463444 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:16.463462 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.464319 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.464323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:16.464488 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:16.465223 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:16.465243 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:16.469505 (+ 4262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":294,"threads_started":1}
W20260811 00:20:16.470036 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:16.459407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:42785 (local address 127.16.71.190:37681)
0811 00:20:16.459599 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:16.459604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:16.460007 (+ 403us) server_negotiation.cc:407] Connection header received
0811 00:20:16.460305 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:16.460311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:16.460406 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:16.460501 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:16.461386 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.462184 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.463649 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.464173 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.464295 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:16.469773 (+ 5478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:42785: BlockingRecv error: recv got EOF from 127.16.71.131:42785 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 00:20:16 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:16 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:16.747917 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:16.737579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:59149)
0811 00:20:16.738654 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:16.738676 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:16.738824 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:16.739238 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:16.739244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:16.739782 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:16.739953 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:16.739963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.741121 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.741126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:16.742099 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:16.742113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.742901 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.742907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:16.743088 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:16.743688 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:16.743706 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:16.747590 (+ 3884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":788,"threads_started":1}
W20260811 00:20:16.748174 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:16.737834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:59149 (local address 127.16.71.190:37681)
0811 00:20:16.738136 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:16.738144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:16.738815 (+ 671us) server_negotiation.cc:407] Connection header received
0811 00:20:16.738953 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:16.738957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:16.739072 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:16.739166 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:16.740115 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.740953 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.742269 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:16.742736 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:16.742855 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:16.747871 (+ 5016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:59149: BlockingRecv error: recv got EOF from 127.16.71.130:59149 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 11 00:20:17 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:17 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:17.107842 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:17.098066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:37871)
0811 00:20:17.098583 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:17.098606 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:17.098748 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:17.099133 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:17.099137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:17.099638 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:17.099828 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:17.099841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.101179 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.101184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:17.102156 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:17.102179 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.102888 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.102892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:17.103074 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:17.103717 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:17.103738 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:17.107576 (+ 3838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":269,"threads_started":1}
W20260811 00:20:17.108098 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:17.098253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37871 (local address 127.16.71.190:37681)
0811 00:20:17.098432 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:17.098437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:17.098686 (+ 249us) server_negotiation.cc:407] Connection header received
0811 00:20:17.098881 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:17.098886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:17.098973 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:17.099099 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:17.100017 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.101029 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.102310 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.102760 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.102860 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:17.107842 (+ 4982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:37871: BlockingRecv error: recv got EOF from 127.16.71.129:37871 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 00:20:17 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:17 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:17.482120 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:17.471137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:50817)
0811 00:20:17.471887 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:17.471905 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:17.472045 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:17.472530 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:17.472534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:17.473265 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:17.473438 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:17.473449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.474979 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.474984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:17.476012 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:17.476028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.476879 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.476885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:17.477086 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:17.477760 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:17.477781 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:17.481863 (+ 4082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":509,"threads_started":1}
W20260811 00:20:17.482554 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:17.471286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:50817 (local address 127.16.71.190:37681)
0811 00:20:17.471533 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:17.471540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:17.472004 (+ 464us) server_negotiation.cc:407] Connection header received
0811 00:20:17.472223 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:17.472228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:17.472302 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:17.472426 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:17.473630 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.474779 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.476187 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.476699 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.476809 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:17.482264 (+ 5455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:50817: BlockingRecv error: recv got EOF from 127.16.71.131:50817 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 00:20:17 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:17 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:17.762071 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:17.749146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:52369)
0811 00:20:17.749690 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:17.749705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:17.749856 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:17.750230 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:17.750235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:17.750697 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:17.750869 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:17.750879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.751953 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.751959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:17.752978 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:17.752992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.753867 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.753874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:17.754064 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:17.757782 (+ 3718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:17.757804 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:17.761711 (+ 3907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":297,"threads_started":1}
W20260811 00:20:17.762295 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:17.749212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:52369 (local address 127.16.71.190:37681)
0811 00:20:17.749530 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:17.749536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:17.749800 (+ 264us) server_negotiation.cc:407] Connection header received
0811 00:20:17.750015 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:17.750020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:17.750086 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:17.750171 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:17.751046 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.751801 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.753127 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:17.753637 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:17.753774 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:17.762063 (+ 8289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:52369: BlockingRecv error: recv got EOF from 127.16.71.130:52369 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 00:20:18 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:18 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:18.118983 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:18.109130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:43609)
0811 00:20:18.109717 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:18.109735 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:18.109869 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:18.110194 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:18.110197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:18.110645 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:18.110828 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:18.110837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.112061 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.112067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:18.113453 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:18.113473 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.114398 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.114404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:18.114608 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:18.115238 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:18.115258 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:18.118715 (+ 3457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":277,"threads_started":1}
W20260811 00:20:18.119308 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:18.109231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:43609 (local address 127.16.71.190:37681)
0811 00:20:18.109505 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:18.109511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:18.109852 (+ 341us) server_negotiation.cc:407] Connection header received
0811 00:20:18.109979 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:18.109983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:18.110054 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:18.110142 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:18.110994 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.111875 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.113670 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.114229 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.114380 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:18.119028 (+ 4648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:43609: BlockingRecv error: recv got EOF from 127.16.71.129:43609 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 00:20:18 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:18 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:18.493762 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:18.483659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:58637)
0811 00:20:18.484290 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:18.484308 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:18.484525 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:18.485102 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:18.485106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:18.485561 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:18.485751 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:18.485761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.486888 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.486892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:18.487845 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:18.487861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.488662 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.488667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:18.488818 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:18.489520 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:18.489540 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:18.493458 (+ 3918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":392,"threads_started":1}
W20260811 00:20:18.494164 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:18.483777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:58637 (local address 127.16.71.190:37681)
0811 00:20:18.484002 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:18.484007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:18.484452 (+ 445us) server_negotiation.cc:407] Connection header received
0811 00:20:18.484743 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:18.484748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:18.484872 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:18.484963 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:18.485953 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.486718 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.488022 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.488484 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.488588 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:18.493825 (+ 5237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:58637: BlockingRecv error: recv got EOF from 127.16.71.131:58637 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 00:20:18 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:18 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:18.773558 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:18.763267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:46533)
0811 00:20:18.764149 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:18.764172 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:18.764344 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:18.764710 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:18.764715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:18.765206 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:18.765410 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:18.765420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.766567 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.766571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:18.767590 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:18.767605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.768397 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.768403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:18.768588 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:18.769328 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:18.769349 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:18.773258 (+ 3909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":484,"threads_started":1}
W20260811 00:20:18.773782 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:18.763449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:46533 (local address 127.16.71.190:37681)
0811 00:20:18.763708 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:18.763713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:18.764269 (+ 556us) server_negotiation.cc:407] Connection header received
0811 00:20:18.764486 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:18.764491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:18.764563 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:18.764681 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:18.765600 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.766403 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.767766 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:18.768215 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:18.768339 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:18.773547 (+ 5208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:46533: BlockingRecv error: recv got EOF from 127.16.71.130:46533 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 11 00:20:19 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:19 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:19.130615 502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:19.120475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:33923)
0811 00:20:19.121087 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:19.121104 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:19.121236 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:19.121540 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:19.121543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:19.121937 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:19.122115 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:19.122125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.123459 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.123464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:19.124704 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:19.124722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.125568 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.125573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:19.125703 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:19.126332 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:19.126356 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:19.130331 (+ 3975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":305,"threads_started":1}
W20260811 00:20:19.130945 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:19.120665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:33923 (local address 127.16.71.190:37681)
0811 00:20:19.120847 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:19.120851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:19.121184 (+ 333us) server_negotiation.cc:407] Connection header received
0811 00:20:19.121335 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:19.121339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:19.121414 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:19.121510 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:19.122322 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.123224 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.124848 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.125372 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.125477 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:19.130691 (+ 5214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:33923: BlockingRecv error: recv got EOF from 127.16.71.129:33923 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 00:20:19 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:19 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:19.504910 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:19.495104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:50815)
0811 00:20:19.495670 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:19.495687 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:19.495909 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:19.496326 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:19.496329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:19.496849 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:19.497053 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:19.497064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.498252 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.498257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:19.499270 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:19.499285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.500113 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.500118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:19.500244 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:19.500968 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:19.500988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:19.504553 (+ 3565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":258,"threads_started":1}
W20260811 00:20:19.505085 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:19.495304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:50815 (local address 127.16.71.190:37681)
0811 00:20:19.495582 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:19.495588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:19.495883 (+ 295us) server_negotiation.cc:407] Connection header received
0811 00:20:19.496104 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:19.496108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:19.496181 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:19.496268 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:19.497227 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.498061 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.499442 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.499929 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.500043 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:19.504836 (+ 4793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:50815: BlockingRecv error: recv got EOF from 127.16.71.131:50815 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 11 00:20:19 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:19 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:19.783847 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:19.774596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:53907)
0811 00:20:19.775135 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:19.775151 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:19.775288 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:19.775607 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:19.775611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:19.776061 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:19.776240 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:19.776250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.777498 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.777502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:19.778446 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:19.778459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.779224 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.779231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:19.779383 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:19.780014 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:19.780033 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:19.783484 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":270,"threads_started":1}
W20260811 00:20:19.783991 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:19.774670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:53907 (local address 127.16.71.190:37681)
0811 00:20:19.774898 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:19.774905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:19.775235 (+ 330us) server_negotiation.cc:407] Connection header received
0811 00:20:19.775406 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:19.775410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:19.775476 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:19.775558 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:19.776396 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.777349 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.778589 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:19.779028 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:19.779143 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:19.783728 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:53907: BlockingRecv error: recv got EOF from 127.16.71.130:53907 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 00:20:20 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:20 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:20.141819 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:20.131802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:48301)
0811 00:20:20.132346 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:20.132368 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:20.132531 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:20.132961 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:20.132966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:20.133470 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:20.133678 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:20.133689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.135177 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.135183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:20.136186 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:20.136201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.137051 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.137057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:20.137207 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:20.137973 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:20.137994 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:20.141556 (+ 3562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":425,"threads_started":1}
W20260811 00:20:20.142154 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:20.132012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:48301 (local address 127.16.71.190:37681)
0811 00:20:20.132223 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:20.132229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:20.132471 (+ 242us) server_negotiation.cc:407] Connection header received
0811 00:20:20.132680 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:20.132685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:20.132774 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:20.132878 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:20.134057 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.135002 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.136355 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.136864 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.136973 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:20.141848 (+ 4875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:48301: BlockingRecv error: recv got EOF from 127.16.71.129:48301 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 00:20:20 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:20 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:20.516392 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:20.506119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:60943)
0811 00:20:20.506743 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:20.506761 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:20.506934 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:20.507343 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:20.507349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:20.507823 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:20.508003 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:20.508014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.509181 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.509185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:20.510164 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:20.510179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.511012 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.511019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:20.511174 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:20.512271 (+ 1097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:20.512291 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:20.516121 (+ 3830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":331,"threads_started":1}
W20260811 00:20:20.516660 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:20.506290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:60943 (local address 127.16.71.190:37681)
0811 00:20:20.506551 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:20.506558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:20.506880 (+ 322us) server_negotiation.cc:407] Connection header received
0811 00:20:20.507093 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:20.507098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:20.507177 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:20.507271 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:20.508184 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.509016 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.510328 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.510802 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.510922 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:20.516391 (+ 5469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:60943: BlockingRecv error: recv got EOF from 127.16.71.131:60943 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 00:20:20 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:20 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:20.795550 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:20.785140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:42243)
0811 00:20:20.785782 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:20.785800 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:20.786006 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:20.786408 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:20.786413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:20.786911 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:20.787077 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:20.787088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.788170 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.788174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:20.789208 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:20.789224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.790072 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.790079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:20.790254 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:20.790958 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:20.790979 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:20.795287 (+ 4308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":342,"threads_started":1}
W20260811 00:20:20.795845 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:20.785303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:42243 (local address 127.16.71.190:37681)
0811 00:20:20.785603 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:20.785609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:20.785948 (+ 339us) server_negotiation.cc:407] Connection header received
0811 00:20:20.786142 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:20.786148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:20.786221 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:20.786309 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:20.787251 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.787994 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.789389 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:20.789894 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:20.790017 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:20.795517 (+ 5500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:42243: BlockingRecv error: recv got EOF from 127.16.71.130:42243 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 11 00:20:21 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:21 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:21.153118 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:21.142888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:49021)
0811 00:20:21.143559 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:21.143577 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:21.143711 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:21.144120 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:21.144124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:21.144558 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:21.144795 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:21.144808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.146030 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.146040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:21.147175 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:21.147192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.148150 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.148155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:21.148298 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:21.149056 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:21.149076 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:21.152798 (+ 3722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":409,"threads_started":1}
W20260811 00:20:21.153369 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:21.142959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:49021 (local address 127.16.71.190:37681)
0811 00:20:21.143163 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:21.143167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:21.143713 (+ 546us) server_negotiation.cc:407] Connection header received
0811 00:20:21.143832 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:21.143837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:21.143918 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:21.144000 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:21.145063 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.145831 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.147375 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.147956 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.148070 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:21.153071 (+ 5001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:49021: BlockingRecv error: recv got EOF from 127.16.71.129:49021 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 00:20:21 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:21 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:21.531658 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:21.517625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:45595)
0811 00:20:21.518224 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:21.518241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:21.518385 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:21.518814 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:21.518819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:21.519376 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:21.519617 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:21.519631 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.520822 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.520826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:21.521824 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:21.521838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.522586 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.522593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:21.522770 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:21.526727 (+ 3957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:21.526754 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:21.531236 (+ 4482us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":280,"threads_started":1}
W20260811 00:20:21.532037 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:21.517811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:45595 (local address 127.16.71.190:37681)
0811 00:20:21.518078 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:21.518084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:21.518333 (+ 249us) server_negotiation.cc:407] Connection header received
0811 00:20:21.518558 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:21.518563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:21.518633 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:21.518719 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:21.519812 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.520665 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.521992 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.522416 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.522547 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:21.531629 (+ 9082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:45595: BlockingRecv error: recv got EOF from 127.16.71.131:45595 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 00:20:21 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:21 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:21.810290 514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:21.796661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:49041)
0811 00:20:21.797414 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:21.797437 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:21.797610 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:21.798097 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:21.798103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:21.798661 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:21.798912 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:21.798941 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.800470 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.800476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:21.802530 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:21.802558 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.804528 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.804534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:21.804721 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:21.805442 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:21.805468 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:21.809919 (+ 4451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":449,"threads_started":1}
W20260811 00:20:21.810406 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:21.796746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:49041 (local address 127.16.71.190:37681)
0811 00:20:21.796979 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:21.796987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:21.797544 (+ 557us) server_negotiation.cc:407] Connection header received
0811 00:20:21.797785 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:21.797792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:21.797884 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:21.797997 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:21.799120 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.800260 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.802847 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:21.803750 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:21.803897 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:21.810174 (+ 6277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:49041: BlockingRecv error: recv got EOF from 127.16.71.130:49041 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 00:20:22 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:22 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:22.164142 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:22.154385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:41033)
0811 00:20:22.155105 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:22.155127 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:22.155281 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:22.155758 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:22.155762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:22.156271 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:22.156464 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:22.156475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.157572 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.157576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:22.158495 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:22.158507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.159220 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.159225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:22.159359 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:22.159997 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:22.160051 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:22.163905 (+ 3854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":302,"threads_started":1}
W20260811 00:20:22.164482 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:22.154600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:41033 (local address 127.16.71.190:37681)
0811 00:20:22.154869 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:22.154875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:22.155222 (+ 347us) server_negotiation.cc:407] Connection header received
0811 00:20:22.155502 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:22.155511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:22.155594 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:22.155678 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:22.156605 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.157455 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.158637 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.159061 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.159161 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:22.164197 (+ 5036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:41033: BlockingRecv error: recv got EOF from 127.16.71.129:41033 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 11 00:20:22 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:22 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:22.543094 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:22.533515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:44351)
0811 00:20:22.534154 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:22.534170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:22.534395 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:22.534800 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:22.534805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:22.535299 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:22.535557 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:22.535569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.536663 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.536669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:22.537657 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:22.537670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.538398 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.538404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:22.538572 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:22.539230 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:22.539248 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:22.542796 (+ 3548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":310,"threads_started":1}
W20260811 00:20:22.543241 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:22.533650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:44351 (local address 127.16.71.190:37681)
0811 00:20:22.533950 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:22.533957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:22.534275 (+ 318us) server_negotiation.cc:407] Connection header received
0811 00:20:22.534547 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:22.534552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:22.534626 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:22.534717 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:22.535706 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.536491 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.537810 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.538232 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.538346 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:22.543023 (+ 4677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:44351: BlockingRecv error: recv got EOF from 127.16.71.131:44351 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 00:20:22 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:22 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:22.826030 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:22.811679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:56977)
0811 00:20:22.812249 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:22.812266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:22.812398 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:22.812765 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:22.812769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:22.813265 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:22.813478 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:22.813489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.814578 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.814582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:22.815630 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:22.815648 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.816601 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.816606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:22.816800 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:22.821851 (+ 5051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:22.821876 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:22.825706 (+ 3830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":344,"threads_started":1}
W20260811 00:20:22.826308 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:22.811833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:56977 (local address 127.16.71.190:37681)
0811 00:20:22.812056 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:22.812061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:22.812337 (+ 276us) server_negotiation.cc:407] Connection header received
0811 00:20:22.812495 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:22.812499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:22.812603 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:22.812716 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:22.813651 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.814426 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.815781 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:22.816412 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:22.816554 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:22.826030 (+ 9476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:56977: BlockingRecv error: recv got EOF from 127.16.71.130:56977 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 00:20:23 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:23 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:23.175002 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:23.165265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:44565)
0811 00:20:23.165971 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:23.165989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:23.166148 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:23.166519 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:23.166524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:23.167016 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:23.167201 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:23.167211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.168282 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.168287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:23.169315 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:23.169332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.170175 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.170182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:23.170348 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:23.171016 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:23.171034 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:23.174747 (+ 3713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":369,"threads_started":1}
W20260811 00:20:23.175177 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:23.165439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:44565 (local address 127.16.71.190:37681)
0811 00:20:23.165699 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:23.165706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:23.166096 (+ 390us) server_negotiation.cc:407] Connection header received
0811 00:20:23.166295 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:23.166299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:23.166368 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:23.166459 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:23.167365 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.168130 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.169495 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.169978 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.170092 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:23.174944 (+ 4852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:44565: BlockingRecv error: recv got EOF from 127.16.71.129:44565 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 00:20:23 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:23 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:23.553871 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:23.544225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:39469)
0811 00:20:23.544828 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:23.544845 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:23.545000 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:23.545487 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:23.545492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:23.545939 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:23.546128 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:23.546138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.547266 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.547271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:23.548400 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:23.548417 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.549230 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.549237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:23.549418 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:23.550076 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:23.550094 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:23.553627 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":390,"threads_started":1}
W20260811 00:20:23.554221 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:23.544405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:39469 (local address 127.16.71.190:37681)
0811 00:20:23.544701 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:23.544707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:23.544941 (+ 234us) server_negotiation.cc:407] Connection header received
0811 00:20:23.545139 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:23.545145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:23.545245 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:23.545337 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:23.546290 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.547112 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.548557 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.549065 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.549183 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:23.553938 (+ 4755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:39469: BlockingRecv error: recv got EOF from 127.16.71.131:39469 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 00:20:23 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:23 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:23.838099 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:23.827451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:44131)
0811 00:20:23.828053 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:23.828071 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:23.828208 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:23.828779 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:23.828784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:23.829399 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:23.829609 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:23.829621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.830848 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.830852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:23.831884 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:23.831901 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.832802 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.832809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:23.833141 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:23.833902 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:23.833927 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:23.837780 (+ 3853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":300,"threads_started":1}
W20260811 00:20:23.838382 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:23.827640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:44131 (local address 127.16.71.190:37681)
0811 00:20:23.827817 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:23.827822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:23.828177 (+ 355us) server_negotiation.cc:407] Connection header received
0811 00:20:23.828480 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:23.828486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:23.828563 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:23.828735 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:23.829791 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.830667 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.832063 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:23.832543 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:23.832716 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:23.838112 (+ 5396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:44131: BlockingRecv error: recv got EOF from 127.16.71.130:44131 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 00:20:24 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:24 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:24.185806 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:24.175991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:50171)
0811 00:20:24.176533 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:24.176549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:24.176698 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:24.177024 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:24.177027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:24.177458 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:24.177649 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:24.177661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.178789 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.178793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:24.179748 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:24.179763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.180573 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.180578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:24.180759 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:24.181352 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:24.181371 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:24.185454 (+ 4083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":286,"threads_started":1}
W20260811 00:20:24.185982 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:24.176120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:50171 (local address 127.16.71.190:37681)
0811 00:20:24.176281 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:24.176285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:24.176655 (+ 370us) server_negotiation.cc:407] Connection header received
0811 00:20:24.176799 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:24.176803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:24.176873 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:24.176994 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:24.177831 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.178644 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.179920 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.180400 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.180506 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:24.185728 (+ 5222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:50171: BlockingRecv error: recv got EOF from 127.16.71.129:50171 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 00:20:24 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:24 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:24.566249 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:24.555169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:52315)
0811 00:20:24.555769 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:24.555786 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:24.555930 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:24.556309 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:24.556312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:24.556787 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:24.557031 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:24.557042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.558238 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.558244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:24.559310 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:24.559326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.560516 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.560524 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:24.560801 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:24.561856 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:24.561890 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:24.565971 (+ 4081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":323,"threads_started":1}
W20260811 00:20:24.566483 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:24.555350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:52315 (local address 127.16.71.190:37681)
0811 00:20:24.555558 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:24.555563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:24.555914 (+ 351us) server_negotiation.cc:407] Connection header received
0811 00:20:24.556103 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:24.556108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:24.556180 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:24.556279 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:24.557220 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.558060 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.559500 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.560124 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.560291 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:24.566262 (+ 5971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:52315: BlockingRecv error: recv got EOF from 127.16.71.131:52315 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 00:20:24 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:24 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:24.849174 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:24.839315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:56531)
0811 00:20:24.839860 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:24.839879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:24.840011 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:24.840412 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:24.840416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:24.840882 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:24.841082 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:24.841094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.842323 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.842328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:24.843438 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:24.843455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.844436 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.844443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:24.844657 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:24.845310 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:24.845332 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:24.848924 (+ 3592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":307,"threads_started":1}
W20260811 00:20:24.849423 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:24.839579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:56531 (local address 127.16.71.190:37681)
0811 00:20:24.839770 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:24.839775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:24.839954 (+ 179us) server_negotiation.cc:407] Connection header received
0811 00:20:24.840133 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:24.840138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:24.840208 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:24.840303 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:24.841266 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.842142 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.843622 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:24.844278 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:24.844402 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:24.849160 (+ 4758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:56531: BlockingRecv error: recv got EOF from 127.16.71.130:56531 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 00:20:25 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:25 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:25.197067 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:25.187259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:45367)
0811 00:20:25.187989 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:25.188006 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:25.188153 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:25.188480 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:25.188483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:25.188931 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:25.189134 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:25.189144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.190216 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.190220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:25.191229 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:25.191245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.192060 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.192066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:25.192235 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:25.192974 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:25.192994 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:25.196743 (+ 3749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":460,"threads_started":1}
W20260811 00:20:25.197315 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:25.187371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:45367 (local address 127.16.71.190:37681)
0811 00:20:25.187592 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:25.187600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:25.188087 (+ 487us) server_negotiation.cc:407] Connection header received
0811 00:20:25.188278 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:25.188282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:25.188348 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:25.188428 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:25.189301 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.190037 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.191434 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.191877 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.191994 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:25.197037 (+ 5043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:45367: BlockingRecv error: recv got EOF from 127.16.71.129:45367 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 00:20:25 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:25 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.131@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:25.578138 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:25.567328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.131:45489)
0811 00:20:25.567900 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:25.567920 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:25.568088 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:25.568426 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:25.568431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:25.568956 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:25.569120 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:25.569130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.570252 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.570259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:25.571320 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:25.571338 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.572329 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.572334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:25.572519 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:25.573351 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:25.573370 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:25.577842 (+ 4472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":305,"threads_started":1}
W20260811 00:20:25.578382 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:25.567523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:45489 (local address 127.16.71.190:37681)
0811 00:20:25.567870 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:25.567876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:25.568011 (+ 135us) server_negotiation.cc:407] Connection header received
0811 00:20:25.568212 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:25.568217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:25.568282 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:25.568367 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:25.569281 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.570085 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.571536 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.572115 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.572259 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:25.578122 (+ 5863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.131:45489: BlockingRecv error: recv got EOF from 127.16.71.131:45489 (error 108)
Metrics: {"server-negotiator.queue_time_us":197}
Aug 11 00:20:25 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:25 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.130@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:25.860342 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:25.850725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.130:50205)
0811 00:20:25.851285 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:25.851302 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:25.851448 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:25.851948 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:25.851953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:25.852430 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:25.852725 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:25.852743 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.853962 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.853966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:25.854981 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:25.854995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.855734 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.855740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:25.855943 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:25.856517 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:25.856535 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:25.860052 (+ 3517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":299,"threads_started":1}
W20260811 00:20:25.860525 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:25.850882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:50205 (local address 127.16.71.190:37681)
0811 00:20:25.851082 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:25.851088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:25.851393 (+ 305us) server_negotiation.cc:407] Connection header received
0811 00:20:25.851579 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:25.851584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:25.851704 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:25.851825 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:25.852879 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.853802 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.855114 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:25.855572 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:25.855669 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:25.860299 (+ 4630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.130:50205: BlockingRecv error: recv got EOF from 127.16.71.130:50205 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 00:20:26 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:20:26 dist-test-slave-f7th krb5kdc[31083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.71.129@KRBTEST.COM for oryx/127.16.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:20:26.209259 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:26.198454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:37681 (local address 127.16.71.129:58933)
0811 00:20:26.199024 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:26.199043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:26.199305 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:26.199688 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:26.199691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:26.200142 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:26.200340 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:26.200350 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.201701 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.201706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:26.202751 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:26.202768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.203599 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.203605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:26.203747 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:26.204475 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:26.204495 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:26.209004 (+ 4509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.71.190:37681: Server oryx/127.16.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":286,"threads_started":1}
W20260811 00:20:26.209488 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:26.198687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:58933 (local address 127.16.71.190:37681)
0811 00:20:26.198947 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:26.198954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:26.199171 (+ 217us) server_negotiation.cc:407] Connection header received
0811 00:20:26.199488 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:26.199494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:26.199568 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:26.199653 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:26.200496 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.201533 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.202923 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.203412 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.203526 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:26.209229 (+ 5703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.71.129:58933: BlockingRecv error: recv got EOF from 127.16.71.129:58933 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 00:20:26.346112 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 32160
I20260811 00:20:26.360759 543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:26.347605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.16.71.129:33669)
0811 00:20:26.348366 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:26.348374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:26.348407 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:20:26.348477 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:26.348485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:26.348768 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:26.349054 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:26.350641 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.351823 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.353304 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.353904 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.354036 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:26.357059 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:26.357071 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:26.357088 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:26.357140 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:26.359901 (+ 2761us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:20:26.360355 (+ 454us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45734: Request ticket server kudu/127.16.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":430,"threads_started":1}
W20260811 00:20:26.360924 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 32160: Not authorized: Client connection negotiation failed: client connection to 127.16.71.129:33669: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:20:26.360986 543 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45734: Request ticket server kudu/127.16.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:20:26.361161 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 32160
I20260811 00:20:26.397735 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 32457
I20260811 00:20:26.413043 545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:26.399007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.16.71.130:34651)
0811 00:20:26.399678 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:26.399686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:26.399718 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:20:26.399795 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:26.399802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:26.400079 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:26.400314 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:26.401910 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.403054 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.404649 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.405305 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.405466 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:26.408445 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:26.408458 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:26.408475 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:26.408536 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:26.411795 (+ 3259us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:20:26.412471 (+ 676us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41710: Request ticket server kudu/127.16.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":346,"threads_started":1}
W20260811 00:20:26.413072 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 32457: Not authorized: Client connection negotiation failed: client connection to 127.16.71.130:34651: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:20:26.413172 545 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41710: Request ticket server kudu/127.16.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:20:26.413268 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 32457
I20260811 00:20:26.449527 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 32690
I20260811 00:20:26.464191 546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:26.450694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.16.71.131:36939)
0811 00:20:26.451381 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:26.451392 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:26.451426 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:20:26.451530 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:26.451539 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:26.451902 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:26.452167 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:26.453955 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.455050 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.456532 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:26.457386 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:26.457516 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:26.460463 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:26.460475 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:26.460492 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:26.460570 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:26.463566 (+ 2996us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:20:26.463912 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54648: Request ticket server kudu/127.16.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":312,"threads_started":1}
W20260811 00:20:26.464318 546 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54648: Request ticket server kudu/127.16.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:20:26.464525 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 32690: Not authorized: Client connection negotiation failed: client connection to 127.16.71.131:36939: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:20:26.464823 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 32690
I20260811 00:20:26.495532 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 32035
I20260811 00:20:26.497123 32150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:26.621608 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 32035
2026-08-11T00:20:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22495 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T00:20:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:20:26Z Disabled control of system clock
I20260811 00:20:26.726658 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42457
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:41201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:42457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 00:20:27.105371 557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:27.105755 557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:27.105901 557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:27.114059 557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:20:27.114419 557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:27.114526 557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:27.114616 557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:20:27.114675 557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:20:27.124822 557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:42457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:27.126332 557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:27.128302 557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:27.141778 566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:27.141753 557 server_base.cc:1034] running on GCE node
W20260811 00:20:27.141688 567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:27.141726 572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:27.143151 557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:27.145027 557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:27.146329 557 hybrid_clock.cc:648] HybridClock initialized: now 1786407627146280 us; error 40 us; skew 500 ppm
I20260811 00:20:27.146801 557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:27.149669 557 webserver.cc:460] Webserver started at http://127.16.71.190:35617/ using document root <none> and password file <none>
I20260811 00:20:27.150468 557 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:27.150574 557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:27.150880 557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:27.154299 557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "becb7710ee4943a488410a1a1fc74811"
format_stamp: "Formatted at 2026-08-11 00:20:27 on dist-test-slave-f7th"
server_key: "8e5fb68931cadeb911b8a40513b174c6"
server_key_iv: "d2ff8164a781969a3b6f3dd9721dd8c9"
server_key_version: "encryptionkey@0"
I20260811 00:20:27.155224 557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "becb7710ee4943a488410a1a1fc74811"
format_stamp: "Formatted at 2026-08-11 00:20:27 on dist-test-slave-f7th"
server_key: "8e5fb68931cadeb911b8a40513b174c6"
server_key_iv: "d2ff8164a781969a3b6f3dd9721dd8c9"
server_key_version: "encryptionkey@0"
I20260811 00:20:27.161150 557 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:20:27.164994 578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:27.166795 557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:20:27.166993 557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "becb7710ee4943a488410a1a1fc74811"
format_stamp: "Formatted at 2026-08-11 00:20:27 on dist-test-slave-f7th"
server_key: "8e5fb68931cadeb911b8a40513b174c6"
server_key_iv: "d2ff8164a781969a3b6f3dd9721dd8c9"
server_key_version: "encryptionkey@0"
I20260811 00:20:27.167234 557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:27.184103 557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:27.185097 557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:27.185381 557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:27.203527 557 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:42457
I20260811 00:20:27.203574 666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:42457 every 8 connection(s)
I20260811 00:20:27.205070 557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:20:27.208036 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 557
I20260811 00:20:27.208259 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:20:27.209060 16670 external_mini_cluster.cc:1428] Setting key a4759ca31be0f4933b928e2f399b5eec
I20260811 00:20:27.209391 667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:27.226255 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811: Bootstrap starting.
I20260811 00:20:27.226537 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:27.211219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.16.71.190:42457)
0811 00:20:27.212537 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:27.212554 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:27.212660 (+ 106us) server_negotiation.cc:407] Connection header received
0811 00:20:27.214905 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:27.214944 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:27.215799 (+ 855us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:27.216437 (+ 638us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:27.217191 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:27.219247 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:27.220770 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:27.221849 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:27.222082 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:27.223351 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:27.223366 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:27.223393 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:27.224193 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:27.224303 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:27.224332 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:27.224465 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:27.224736 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 00:20:27.225140 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":420,"threads_started":1}
I20260811 00:20:27.230507 667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:27.231572 667 log.cc:824] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:27.234902 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811: No bootstrap required, opened a new log
I20260811 00:20:27.243242 667 raft_consensus.cc:348] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "becb7710ee4943a488410a1a1fc74811" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42457 } }
I20260811 00:20:27.243713 667 raft_consensus.cc:374] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:27.243801 667 raft_consensus.cc:729] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: becb7710ee4943a488410a1a1fc74811, State: Initialized, Role: FOLLOWER
I20260811 00:20:27.244429 667 consensus_queue.cc:260] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "becb7710ee4943a488410a1a1fc74811" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42457 } }
I20260811 00:20:27.244719 667 raft_consensus.cc:388] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:20:27.244837 667 raft_consensus.cc:482] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:20:27.244975 667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:27.247292 667 raft_consensus.cc:504] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "becb7710ee4943a488410a1a1fc74811" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42457 } }
I20260811 00:20:27.247866 667 leader_election.cc:302] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: becb7710ee4943a488410a1a1fc74811; no voters:
I20260811 00:20:27.248955 667 leader_election.cc:288] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:20:27.249120 676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:27.251762 667 sys_catalog.cc:564] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:20:27.252818 676 raft_consensus.cc:686] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [term 1 LEADER]: Becoming Leader. State: Replica: becb7710ee4943a488410a1a1fc74811, State: Running, Role: LEADER
I20260811 00:20:27.253543 676 consensus_queue.cc:237] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "becb7710ee4943a488410a1a1fc74811" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42457 } }
I20260811 00:20:27.257304 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [sys.catalog]: SysCatalogTable state changed. Reason: New leader becb7710ee4943a488410a1a1fc74811. Latest consensus state: current_term: 1 leader_uuid: "becb7710ee4943a488410a1a1fc74811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "becb7710ee4943a488410a1a1fc74811" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42457 } } }
I20260811 00:20:27.257328 679 sys_catalog.cc:455] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "becb7710ee4943a488410a1a1fc74811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "becb7710ee4943a488410a1a1fc74811" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42457 } } }
I20260811 00:20:27.257977 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:27.258102 679 sys_catalog.cc:458] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811 [sys.catalog]: This master's current role is: LEADER
W20260811 00:20:27.267880 712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:20:27.268077 712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:20:27.268254 699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:20:27.270833 699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:20:27.281807 699 catalog_manager.cc:1347] Generated new cluster ID: e4b2e027150b4c84a4f78fdef2868228
I20260811 00:20:27.281965 699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:20:27.294478 699 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:20:27.295914 699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:20:27.306851 699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P becb7710ee4943a488410a1a1fc74811: Generated new TSK 0
I20260811 00:20:27.307832 699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:20:27.320044 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42457
--builtin_ntp_servers=127.16.71.148:41201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 00:20:27.732365 716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:27.732733 716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:27.732872 716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:27.740832 716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:27.740967 716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:27.741104 716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:20:27.751151 716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:27.752658 716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:27.754796 716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:27.769979 723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:27.770052 724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:27.769979 731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:27.770684 716 server_base.cc:1034] running on GCE node
I20260811 00:20:27.771538 716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:27.772792 716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:27.774060 716 hybrid_clock.cc:648] HybridClock initialized: now 1786407627773974 us; error 60 us; skew 500 ppm
I20260811 00:20:27.774469 716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:27.777282 716 webserver.cc:460] Webserver started at http://127.16.71.129:35665/ using document root <none> and password file <none>
I20260811 00:20:27.778116 716 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:27.778236 716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:27.778592 716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:27.781999 716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "42487ee315a44cd493e83aaacc1e58c1"
format_stamp: "Formatted at 2026-08-11 00:20:27 on dist-test-slave-f7th"
server_key: "133f157df014ee84ad57747ee09b5c0c"
server_key_iv: "5cb2c56a3c427548ea4a48a7148eecda"
server_key_version: "encryptionkey@0"
I20260811 00:20:27.782953 716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "42487ee315a44cd493e83aaacc1e58c1"
format_stamp: "Formatted at 2026-08-11 00:20:27 on dist-test-slave-f7th"
server_key: "133f157df014ee84ad57747ee09b5c0c"
server_key_iv: "5cb2c56a3c427548ea4a48a7148eecda"
server_key_version: "encryptionkey@0"
I20260811 00:20:27.789687 716 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:20:27.794373 745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:27.796048 716 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:20:27.796341 716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "42487ee315a44cd493e83aaacc1e58c1"
format_stamp: "Formatted at 2026-08-11 00:20:27 on dist-test-slave-f7th"
server_key: "133f157df014ee84ad57747ee09b5c0c"
server_key_iv: "5cb2c56a3c427548ea4a48a7148eecda"
server_key_version: "encryptionkey@0"
I20260811 00:20:27.796693 716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:27.819028 716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:27.819983 716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:27.820290 716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:27.821410 716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:27.823061 716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:27.823169 716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:27.823282 716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:27.823351 716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:27.857398 716 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:33039
I20260811 00:20:27.857460 956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:33039 every 8 connection(s)
I20260811 00:20:27.858778 716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:20:27.867223 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 716
I20260811 00:20:27.867467 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:20:27.868353 16670 external_mini_cluster.cc:1428] Setting key 39153f57da3ec4ae877d5e54cab17626
I20260811 00:20:27.872017 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42457
--builtin_ntp_servers=127.16.71.148:41201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 00:20:27.878288 961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:27.861505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:36797 (local address 127.16.71.190:42457)
0811 00:20:27.862037 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:27.862041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:27.863506 (+ 1465us) server_negotiation.cc:407] Connection header received
0811 00:20:27.865484 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:27.865489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:27.865569 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:27.865684 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:27.867002 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:27.868136 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:27.870265 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:27.870835 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:27.870958 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:27.877256 (+ 6298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:27.877263 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:27.877268 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:27.877295 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:27.877352 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:27.877372 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:27.877487 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:27.877784 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 00:20:27.877916 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":273,"threads_started":1}
I20260811 00:20:27.879426 962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:27.862201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42457 (local address 127.16.71.129:36797)
0811 00:20:27.863310 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:27.863360 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:27.865144 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:27.865933 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:27.865944 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:27.866049 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:27.866741 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:27.866763 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:27.868367 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:27.868371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:27.870059 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:27.870077 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:27.871182 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:27.871192 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:27.871593 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:27.876773 (+ 5180us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:20:27.876776 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:27.876833 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:20:27.876956 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:27.877529 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:27.877542 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:27.877553 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:20:27.877732 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 00:20:27.878123 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":366,"threads_started":1}
I20260811 00:20:27.881204 957 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42457
I20260811 00:20:27.881605 957 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:27.882638 957 heartbeater.cc:511] Master 127.16.71.190:42457 requested a full tablet report, sending...
I20260811 00:20:27.885201 607 ts_manager.cc:194] Registered new tserver with Master: 42487ee315a44cd493e83aaacc1e58c1 (127.16.71.129:33039)
I20260811 00:20:27.886804 607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.129:36797
W20260811 00:20:28.239588 963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:28.239948 963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:28.240048 963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:28.248003 963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:28.248147 963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:28.248291 963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:20:28.258481 963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:28.260071 963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:28.262149 963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:28.276604 983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:28.276672 988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:28.276701 978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:28.277405 963 server_base.cc:1034] running on GCE node
I20260811 00:20:28.278173 963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:28.279222 963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:28.280464 963 hybrid_clock.cc:648] HybridClock initialized: now 1786407628280389 us; error 65 us; skew 500 ppm
I20260811 00:20:28.280858 963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:28.283388 963 webserver.cc:460] Webserver started at http://127.16.71.130:40575/ using document root <none> and password file <none>
I20260811 00:20:28.284145 963 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:28.284256 963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:28.284680 963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:28.287923 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "093e351fd02442c7ac912ebaf20eedaa"
format_stamp: "Formatted at 2026-08-11 00:20:28 on dist-test-slave-f7th"
server_key: "316b002ebad0c5f5768992402cdcb741"
server_key_iv: "85bce2e26505360b192077b76ec6b05e"
server_key_version: "encryptionkey@0"
I20260811 00:20:28.288908 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "093e351fd02442c7ac912ebaf20eedaa"
format_stamp: "Formatted at 2026-08-11 00:20:28 on dist-test-slave-f7th"
server_key: "316b002ebad0c5f5768992402cdcb741"
server_key_iv: "85bce2e26505360b192077b76ec6b05e"
server_key_version: "encryptionkey@0"
I20260811 00:20:28.295065 963 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:20:28.299000 997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:28.300446 963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:20:28.300689 963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "093e351fd02442c7ac912ebaf20eedaa"
format_stamp: "Formatted at 2026-08-11 00:20:28 on dist-test-slave-f7th"
server_key: "316b002ebad0c5f5768992402cdcb741"
server_key_iv: "85bce2e26505360b192077b76ec6b05e"
server_key_version: "encryptionkey@0"
I20260811 00:20:28.300933 963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:28.323884 963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:28.324872 963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:28.325165 963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:28.326225 963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:28.327873 963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:28.327970 963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:28.328059 963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:28.328115 963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:28.361629 963 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:42599
I20260811 00:20:28.361677 1193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:42599 every 8 connection(s)
I20260811 00:20:28.363036 963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:20:28.364290 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 963
I20260811 00:20:28.364511 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:20:28.365110 16670 external_mini_cluster.cc:1428] Setting key 1b412a0490faefdf5ca3b86a06f69d6b
I20260811 00:20:28.368571 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42457
--builtin_ntp_servers=127.16.71.148:41201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 00:20:28.379308 961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:28.366511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:55941 (local address 127.16.71.190:42457)
0811 00:20:28.366722 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:28.366729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:28.368635 (+ 1906us) server_negotiation.cc:407] Connection header received
0811 00:20:28.371266 (+ 2631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:28.371271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:28.371360 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:28.371454 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:28.372987 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.373753 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.376117 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.376582 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.376755 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:28.378506 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:28.378511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:28.378515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:28.378538 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:28.378574 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:28.378587 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:28.378663 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:28.378916 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 00:20:28.379045 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:20:28.380808 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:28.367161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42457 (local address 127.16.71.130:55941)
0811 00:20:28.368419 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:28.368472 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:28.370836 (+ 2364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:28.371754 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:28.371767 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:28.371907 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:28.372771 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:28.372799 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.373960 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.373965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:28.375838 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:28.375870 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.376890 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.376899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:28.377218 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:28.378054 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:20:28.378058 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:28.378123 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:20:28.378254 (+ 131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:28.378685 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:28.378696 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:28.378703 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:20:28.378914 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 00:20:28.379366 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":362,"threads_started":1}
I20260811 00:20:28.382511 1194 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42457
I20260811 00:20:28.382876 1194 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:28.383751 1194 heartbeater.cc:511] Master 127.16.71.190:42457 requested a full tablet report, sending...
I20260811 00:20:28.385547 607 ts_manager.cc:194] Registered new tserver with Master: 093e351fd02442c7ac912ebaf20eedaa (127.16.71.130:42599)
I20260811 00:20:28.386391 607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.130:55941
W20260811 00:20:28.736356 1198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:28.736726 1198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:28.736869 1198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:28.744572 1198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:28.744735 1198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:28.744882 1198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:20:28.755162 1198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:42457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:28.756599 1198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:28.758685 1198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:28.773788 1206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:28.773727 1205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:28.773820 1209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:28.774855 1198 server_base.cc:1034] running on GCE node
I20260811 00:20:28.775614 1198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:28.776762 1198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:28.778005 1198 hybrid_clock.cc:648] HybridClock initialized: now 1786407628777934 us; error 60 us; skew 500 ppm
I20260811 00:20:28.778373 1198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:28.780900 1198 webserver.cc:460] Webserver started at http://127.16.71.131:35049/ using document root <none> and password file <none>
I20260811 00:20:28.781665 1198 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:28.781785 1198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:28.782140 1198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:28.785423 1198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "05377c52121742d4af3102b0c4b6efa5"
format_stamp: "Formatted at 2026-08-11 00:20:28 on dist-test-slave-f7th"
server_key: "dd1cb565c41053ec1a28ebc5fd6bf6db"
server_key_iv: "41d8a534aa0a482e357467227c647117"
server_key_version: "encryptionkey@0"
I20260811 00:20:28.786309 1198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "05377c52121742d4af3102b0c4b6efa5"
format_stamp: "Formatted at 2026-08-11 00:20:28 on dist-test-slave-f7th"
server_key: "dd1cb565c41053ec1a28ebc5fd6bf6db"
server_key_iv: "41d8a534aa0a482e357467227c647117"
server_key_version: "encryptionkey@0"
I20260811 00:20:28.792551 1198 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:20:28.796851 1215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:28.798511 1198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:20:28.798722 1198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "05377c52121742d4af3102b0c4b6efa5"
format_stamp: "Formatted at 2026-08-11 00:20:28 on dist-test-slave-f7th"
server_key: "dd1cb565c41053ec1a28ebc5fd6bf6db"
server_key_iv: "41d8a534aa0a482e357467227c647117"
server_key_version: "encryptionkey@0"
I20260811 00:20:28.798967 1198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:28.818126 1198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:28.819121 1198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:28.819463 1198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:28.820542 1198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:28.822299 1198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:28.822432 1198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:28.822570 1198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:28.822656 1198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:28.857995 1198 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:46535
I20260811 00:20:28.858018 1421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:46535 every 8 connection(s)
I20260811 00:20:28.859413 1198 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:20:28.864717 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 1198
I20260811 00:20:28.864908 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:20:28.865372 16670 external_mini_cluster.cc:1428] Setting key f7369f4fee3a79c63002c1efd741dcf1
I20260811 00:20:28.874152 961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:28.862506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:36631 (local address 127.16.71.190:42457)
0811 00:20:28.862733 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:28.862739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:28.864141 (+ 1402us) server_negotiation.cc:407] Connection header received
0811 00:20:28.866291 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:28.866297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:28.866396 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:28.866495 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:28.868123 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.869160 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.870971 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.871699 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.871838 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:28.873335 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:28.873340 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:28.873343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:28.873365 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:28.873401 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:28.873413 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:28.873484 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:28.873756 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 00:20:28.873890 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:20:28.875140 1426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:28.862989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:42457 (local address 127.16.71.131:36631)
0811 00:20:28.863964 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:28.864003 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:28.865888 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:28.866850 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:28.866865 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:28.867055 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:28.867920 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:28.867943 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.869354 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.869359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:28.870745 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:28.870770 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.871979 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.871989 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:28.872306 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:28.872993 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:20:28.872996 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:28.873040 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:20:28.873133 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:28.873502 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:28.873511 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:28.873518 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:20:28.873680 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 00:20:28.874038 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":302,"threads_started":1}
I20260811 00:20:28.876693 1424 heartbeater.cc:347] Connected to a master server at 127.16.71.190:42457
I20260811 00:20:28.877039 1424 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:28.877914 1424 heartbeater.cc:511] Master 127.16.71.190:42457 requested a full tablet report, sending...
I20260811 00:20:28.879700 607 ts_manager.cc:194] Registered new tserver with Master: 05377c52121742d4af3102b0c4b6efa5 (127.16.71.131:46535)
I20260811 00:20:28.880574 607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.131:36631
I20260811 00:20:28.881434 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:20:28.888669 961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:28.886895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.16.71.190:42457)
0811 00:20:28.887097 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:28.887105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:28.887618 (+ 513us) server_negotiation.cc:407] Connection header received
0811 00:20:28.887795 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:28.887800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:28.887868 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:28.887959 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:28.888427 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35600: BlockingRecv error: recv got EOF from 127.0.0.1:35600 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:20:28.889595 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 716
I20260811 00:20:28.890391 957 heartbeater.cc:503] Master 127.16.71.190:42457 was elected leader, sending a full tablet report...
I20260811 00:20:28.898260 1442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:28.890585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.16.71.129:33039)
0811 00:20:28.891191 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:28.891200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:28.891234 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:20:28.891299 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:28.891306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:28.891584 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:28.891771 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:28.892183 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.893424 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.894769 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:28.895389 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:28.895513 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:28.896741 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:28.896751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:28.896767 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:28.897381 (+ 614us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:28.897435 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:28.897457 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:28.897567 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:28.897720 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 00:20:28.897849 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":283,"threads_started":1}
I20260811 00:20:28.899596 951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:28.984854 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 716
I20260811 00:20:29.006091 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 963
I20260811 00:20:29.014389 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:29.007104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.16.71.130:42599)
0811 00:20:29.007671 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:29.007678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:29.007706 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:20:29.007771 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:29.007778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:29.007998 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:29.008136 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:29.008483 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:29.009572 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:29.010927 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:29.011557 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:29.011674 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:29.012799 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:29.012809 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:29.012825 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:29.013474 (+ 649us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:29.013552 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:29.013578 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:29.013678 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:29.013872 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 00:20:29.014003 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":271,"threads_started":1}
I20260811 00:20:29.015847 1186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:29.099221 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 963
I20260811 00:20:29.117786 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 1198
I20260811 00:20:29.127609 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:29.119103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.16.71.131:46535)
0811 00:20:29.119932 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:29.119980 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:29.120017 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:20:29.120084 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:29.120091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:29.120328 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:29.120479 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:29.121055 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:29.122189 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:29.123578 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:29.124256 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:29.124377 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:29.125768 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:29.125779 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:29.125797 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:29.126557 (+ 760us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:29.126617 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:29.126644 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:29.126766 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:29.127074 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 00:20:29.127241 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":366,"threads_started":1}
I20260811 00:20:29.129070 1416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:29.218681 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 1198
I20260811 00:20:29.242100 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 557
I20260811 00:20:29.243487 660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:29.351959 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 557
2026-08-11T00:20:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2723 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T00:20:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:20:29Z Disabled control of system clock
I20260811 00:20:29.451136 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:38723
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:38723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 00:20:29.828712 1460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:29.829085 1460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:29.829226 1460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:29.837513 1460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:20:29.837653 1460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:29.837707 1460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:29.837791 1460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:20:29.837864 1460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:20:29.847877 1460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:38723
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:38723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:29.849432 1460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:29.851544 1460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:29.864645 1473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:29.864540 1474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:29.864586 1479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:29.865610 1460 server_base.cc:1034] running on GCE node
I20260811 00:20:29.866631 1460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:29.868417 1460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:29.869717 1460 hybrid_clock.cc:648] HybridClock initialized: now 1786407629869642 us; error 69 us; skew 500 ppm
I20260811 00:20:29.870172 1460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:29.872845 1460 webserver.cc:460] Webserver started at http://127.16.71.190:40911/ using document root <none> and password file <none>
I20260811 00:20:29.873660 1460 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:29.873780 1460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:29.874136 1460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:29.877559 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "d35c8fc4659d4c629df9c553f713b769"
format_stamp: "Formatted at 2026-08-11 00:20:29 on dist-test-slave-f7th"
server_key: "19b80859178994bfb53872ef173a6db4"
server_key_iv: "3b2a39aaeeee9667f6bb5db5a4f08443"
server_key_version: "encryptionkey@0"
I20260811 00:20:29.878594 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "d35c8fc4659d4c629df9c553f713b769"
format_stamp: "Formatted at 2026-08-11 00:20:29 on dist-test-slave-f7th"
server_key: "19b80859178994bfb53872ef173a6db4"
server_key_iv: "3b2a39aaeeee9667f6bb5db5a4f08443"
server_key_version: "encryptionkey@0"
I20260811 00:20:29.885102 1460 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:20:29.889150 1490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:29.890687 1460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:20:29.890900 1460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "d35c8fc4659d4c629df9c553f713b769"
format_stamp: "Formatted at 2026-08-11 00:20:29 on dist-test-slave-f7th"
server_key: "19b80859178994bfb53872ef173a6db4"
server_key_iv: "3b2a39aaeeee9667f6bb5db5a4f08443"
server_key_version: "encryptionkey@0"
I20260811 00:20:29.891158 1460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:29.897711 1460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:29.898622 1460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:29.898939 1460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:29.917676 1460 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:38723
I20260811 00:20:29.917716 1572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:38723 every 8 connection(s)
I20260811 00:20:29.919255 1460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:20:29.923027 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 1460
I20260811 00:20:29.923249 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:20:29.923699 1573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:29.923969 16670 external_mini_cluster.cc:1428] Setting key 339222733da3be959f1258c53d10479e
I20260811 00:20:29.933039 1580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:29.926181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.16.71.190:38723)
0811 00:20:29.927367 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:29.927378 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:29.927426 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:20:29.929038 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:29.929065 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:29.929561 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:29.929959 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:29.931215 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:29.931222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:29.931237 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:29.931256 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:29.931336 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:29.931353 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:29.931556 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:29.931661 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 00:20:29.932013 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":432,"threads_started":1}
I20260811 00:20:29.938591 1573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769: Bootstrap starting.
I20260811 00:20:29.942291 1573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:29.943455 1573 log.cc:824] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:29.946882 1573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769: No bootstrap required, opened a new log
I20260811 00:20:29.954972 1573 raft_consensus.cc:348] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35c8fc4659d4c629df9c553f713b769" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38723 } }
I20260811 00:20:29.955427 1573 raft_consensus.cc:374] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:29.955530 1573 raft_consensus.cc:729] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35c8fc4659d4c629df9c553f713b769, State: Initialized, Role: FOLLOWER
I20260811 00:20:29.956174 1573 consensus_queue.cc:260] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35c8fc4659d4c629df9c553f713b769" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38723 } }
I20260811 00:20:29.956450 1573 raft_consensus.cc:388] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:20:29.956584 1573 raft_consensus.cc:482] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:20:29.956785 1573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:29.959194 1573 raft_consensus.cc:504] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35c8fc4659d4c629df9c553f713b769" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38723 } }
I20260811 00:20:29.959856 1573 leader_election.cc:302] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d35c8fc4659d4c629df9c553f713b769; no voters:
I20260811 00:20:29.960593 1573 leader_election.cc:288] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:20:29.960799 1583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:29.961949 1583 raft_consensus.cc:686] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [term 1 LEADER]: Becoming Leader. State: Replica: d35c8fc4659d4c629df9c553f713b769, State: Running, Role: LEADER
I20260811 00:20:29.962560 1583 consensus_queue.cc:237] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35c8fc4659d4c629df9c553f713b769" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38723 } }
I20260811 00:20:29.963014 1573 sys_catalog.cc:564] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:20:29.965577 1584 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d35c8fc4659d4c629df9c553f713b769" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35c8fc4659d4c629df9c553f713b769" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38723 } } }
I20260811 00:20:29.965610 1585 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35c8fc4659d4c629df9c553f713b769. Latest consensus state: current_term: 1 leader_uuid: "d35c8fc4659d4c629df9c553f713b769" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35c8fc4659d4c629df9c553f713b769" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38723 } } }
I20260811 00:20:29.966002 1584 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:29.966044 1585 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:29.968554 1589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:20:29.974205 1589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:20:29.984289 1589 catalog_manager.cc:1347] Generated new cluster ID: 13b738be69534a6db45bee4a433fbf60
I20260811 00:20:29.984436 1589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:20:30.008934 1589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:20:30.009548 1589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:20:30.020967 1589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d35c8fc4659d4c629df9c553f713b769: Generated new TSK 0
I20260811 00:20:30.022399 1589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:20:30.033427 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38723
--builtin_ntp_servers=127.16.71.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 00:20:30.401674 1609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:30.402143 1609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:30.402307 1609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:30.412379 1609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:30.412510 1609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:30.412750 1609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:20:30.423900 1609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:30.425736 1609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:30.427729 1609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:30.442799 1619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:30.442775 1626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:30.442768 1624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:30.446696 1609 server_base.cc:1034] running on GCE node
I20260811 00:20:30.447501 1609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:30.448698 1609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:30.449971 1609 hybrid_clock.cc:648] HybridClock initialized: now 1786407630449910 us; error 42 us; skew 500 ppm
I20260811 00:20:30.450398 1609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:30.453434 1609 webserver.cc:460] Webserver started at http://127.16.71.129:43045/ using document root <none> and password file <none>
I20260811 00:20:30.454252 1609 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:30.454373 1609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:30.454672 1609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:30.458577 1609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "104dca49a762437784bcf0c159fe7a61"
format_stamp: "Formatted at 2026-08-11 00:20:30 on dist-test-slave-f7th"
server_key: "c70f1363e738b6da9d6069c0026f38f0"
server_key_iv: "2bfd58b8d631a7aaf7856d3cb50363a8"
server_key_version: "encryptionkey@0"
I20260811 00:20:30.459597 1609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "104dca49a762437784bcf0c159fe7a61"
format_stamp: "Formatted at 2026-08-11 00:20:30 on dist-test-slave-f7th"
server_key: "c70f1363e738b6da9d6069c0026f38f0"
server_key_iv: "2bfd58b8d631a7aaf7856d3cb50363a8"
server_key_version: "encryptionkey@0"
I20260811 00:20:30.465649 1609 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:20:30.469859 1639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:30.471405 1609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:20:30.471606 1609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "104dca49a762437784bcf0c159fe7a61"
format_stamp: "Formatted at 2026-08-11 00:20:30 on dist-test-slave-f7th"
server_key: "c70f1363e738b6da9d6069c0026f38f0"
server_key_iv: "2bfd58b8d631a7aaf7856d3cb50363a8"
server_key_version: "encryptionkey@0"
I20260811 00:20:30.471951 1609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:30.478610 1609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:30.479560 1609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:30.479835 1609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:30.480932 1609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:30.482558 1609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:30.482666 1609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:30.482782 1609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:30.482852 1609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:30.516289 1609 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:34007
I20260811 00:20:30.516359 1833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:34007 every 8 connection(s)
I20260811 00:20:30.517720 1609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:20:30.524531 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 1609
I20260811 00:20:30.524765 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:20:30.525642 16670 external_mini_cluster.cc:1428] Setting key ed253949cd129cf0b74a43ea284512da
I20260811 00:20:30.527508 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:30.520715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:49379 (local address 127.16.71.190:38723)
0811 00:20:30.521267 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:30.521271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:30.522656 (+ 1385us) server_negotiation.cc:407] Connection header received
0811 00:20:30.524569 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:30.524575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:30.524682 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:30.524803 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:30.526582 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:30.526589 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:30.526594 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:30.526619 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:30.526674 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:30.526689 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:30.526826 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:30.527102 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 00:20:30.527168 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":276,"threads_started":1}
I20260811 00:20:30.528707 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:30.521334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:38723 (local address 127.16.71.129:49379)
0811 00:20:30.522464 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:30.522509 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:30.524232 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:30.525163 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:30.525176 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:30.525336 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:30.526156 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:20:30.526159 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:30.526219 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:20:30.526366 (+ 147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:30.526830 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:30.526843 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:30.526852 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:20:30.527135 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 00:20:30.527367 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":324,"threads_started":1}
I20260811 00:20:30.529265 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38723
--builtin_ntp_servers=127.16.71.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 00:20:30.530476 1834 heartbeater.cc:347] Connected to a master server at 127.16.71.190:38723
I20260811 00:20:30.530853 1834 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:30.531621 1834 heartbeater.cc:511] Master 127.16.71.190:38723 requested a full tablet report, sending...
I20260811 00:20:30.534231 1519 ts_manager.cc:194] Registered new tserver with Master: 104dca49a762437784bcf0c159fe7a61 (127.16.71.129:34007)
W20260811 00:20:30.891170 1844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:30.891510 1844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:30.891609 1844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:30.899516 1844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:30.899642 1844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:30.899762 1844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:20:30.909615 1844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:30.910980 1844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:30.912895 1844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:30.927367 1850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:30.927367 1852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:30.927503 1858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:30.928963 1844 server_base.cc:1034] running on GCE node
I20260811 00:20:30.929751 1844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:30.930856 1844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:30.932145 1844 hybrid_clock.cc:648] HybridClock initialized: now 1786407630932080 us; error 54 us; skew 500 ppm
I20260811 00:20:30.932524 1844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:30.935145 1844 webserver.cc:460] Webserver started at http://127.16.71.130:41413/ using document root <none> and password file <none>
I20260811 00:20:30.936007 1844 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:30.936156 1844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:30.936507 1844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:30.939940 1844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "803826fefa4341a09ddab00d8d4e4d8a"
format_stamp: "Formatted at 2026-08-11 00:20:30 on dist-test-slave-f7th"
server_key: "11f7eeb014c29336aa8a339dd773af11"
server_key_iv: "fa31a6372011b403837292cdd33e1985"
server_key_version: "encryptionkey@0"
I20260811 00:20:30.940956 1844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "803826fefa4341a09ddab00d8d4e4d8a"
format_stamp: "Formatted at 2026-08-11 00:20:30 on dist-test-slave-f7th"
server_key: "11f7eeb014c29336aa8a339dd773af11"
server_key_iv: "fa31a6372011b403837292cdd33e1985"
server_key_version: "encryptionkey@0"
I20260811 00:20:30.947456 1844 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:20:30.953511 1866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:30.955389 1844 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:20:30.955628 1844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "803826fefa4341a09ddab00d8d4e4d8a"
format_stamp: "Formatted at 2026-08-11 00:20:30 on dist-test-slave-f7th"
server_key: "11f7eeb014c29336aa8a339dd773af11"
server_key_iv: "fa31a6372011b403837292cdd33e1985"
server_key_version: "encryptionkey@0"
I20260811 00:20:30.955884 1844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:30.962432 1844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:30.963555 1844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:30.963832 1844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:30.964887 1844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:30.966444 1844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:30.966539 1844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:30.966655 1844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:30.966724 1844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:31.002383 1844 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:37169
I20260811 00:20:31.002460 2071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:37169 every 8 connection(s)
I20260811 00:20:31.004127 1844 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:20:31.011860 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 1844
I20260811 00:20:31.012073 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:20:31.012656 16670 external_mini_cluster.cc:1428] Setting key 3bddc49a3ee8b91c80a019b7fd59853b
I20260811 00:20:31.014295 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:31.007394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:49017 (local address 127.16.71.190:38723)
0811 00:20:31.007652 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:31.007657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:31.009284 (+ 1627us) server_negotiation.cc:407] Connection header received
0811 00:20:31.011172 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:31.011177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:31.011246 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:31.011353 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:31.013363 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:31.013371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:31.013375 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:31.013406 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:31.013458 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:31.013477 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:31.013658 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:31.013903 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 00:20:31.013991 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 00:20:31.016055 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:31.007930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:38723 (local address 127.16.71.130:49017)
0811 00:20:31.009062 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:31.009106 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:31.010812 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:31.011734 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:31.011749 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:31.011905 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:31.012871 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:20:31.012875 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:31.012939 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:20:31.013084 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:31.013602 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:31.013615 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:31.013624 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:20:31.013961 (+ 337us) client_negotiation.cc:240] Negotiation successful
0811 00:20:31.014236 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":303,"threads_started":1}
I20260811 00:20:31.016561 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38723
--builtin_ntp_servers=127.16.71.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 00:20:31.017975 2072 heartbeater.cc:347] Connected to a master server at 127.16.71.190:38723
I20260811 00:20:31.018330 2072 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:31.018999 2072 heartbeater.cc:511] Master 127.16.71.190:38723 requested a full tablet report, sending...
I20260811 00:20:31.020843 1519 ts_manager.cc:194] Registered new tserver with Master: 803826fefa4341a09ddab00d8d4e4d8a (127.16.71.130:37169)
W20260811 00:20:31.378249 2082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:31.378582 2082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:31.378702 2082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:31.386226 2082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:31.386344 2082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:31.386502 2082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:20:31.396106 2082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:31.397559 2082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:31.399529 2082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:31.413539 2091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:31.413504 2095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:31.413551 2090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:31.414839 2082 server_base.cc:1034] running on GCE node
I20260811 00:20:31.415494 2082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:31.416575 2082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:31.417852 2082 hybrid_clock.cc:648] HybridClock initialized: now 1786407631417791 us; error 40 us; skew 500 ppm
I20260811 00:20:31.418220 2082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:20:31.420719 2082 webserver.cc:460] Webserver started at http://127.16.71.131:43563/ using document root <none> and password file <none>
I20260811 00:20:31.421468 2082 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:31.421581 2082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:31.421909 2082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:31.425056 2082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "e8a1d66c55ed42acb94ca132bcb7b647"
format_stamp: "Formatted at 2026-08-11 00:20:31 on dist-test-slave-f7th"
server_key: "63cf66081749a5c08b508c9750dfa005"
server_key_iv: "874e00cf12392368e2434dfe1a1aaf1d"
server_key_version: "encryptionkey@0"
I20260811 00:20:31.426061 2082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "e8a1d66c55ed42acb94ca132bcb7b647"
format_stamp: "Formatted at 2026-08-11 00:20:31 on dist-test-slave-f7th"
server_key: "63cf66081749a5c08b508c9750dfa005"
server_key_iv: "874e00cf12392368e2434dfe1a1aaf1d"
server_key_version: "encryptionkey@0"
I20260811 00:20:31.432138 2082 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:20:31.436192 2113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:31.437928 2082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:20:31.438146 2082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "e8a1d66c55ed42acb94ca132bcb7b647"
format_stamp: "Formatted at 2026-08-11 00:20:31 on dist-test-slave-f7th"
server_key: "63cf66081749a5c08b508c9750dfa005"
server_key_iv: "874e00cf12392368e2434dfe1a1aaf1d"
server_key_version: "encryptionkey@0"
I20260811 00:20:31.438387 2082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:31.444880 2082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:31.445883 2082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:31.446164 2082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:31.447355 2082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:31.449107 2082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:31.449213 2082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:31.449306 2082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:31.449358 2082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:31.482192 2082 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:37555
I20260811 00:20:31.482245 2310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:37555 every 8 connection(s)
I20260811 00:20:31.483595 2082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:20:31.488173 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 2082
I20260811 00:20:31.488355 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:20:31.489055 16670 external_mini_cluster.cc:1428] Setting key 49e54c223d638feaa17aa6bd7af58a2f
I20260811 00:20:31.493096 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:31.486545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:38835 (local address 127.16.71.190:38723)
0811 00:20:31.486770 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:31.486776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:31.488551 (+ 1775us) server_negotiation.cc:407] Connection header received
0811 00:20:31.490732 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:31.490736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:31.490794 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:31.490879 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:31.492387 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:31.492392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:31.492395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:31.492413 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:31.492447 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:31.492459 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:31.492571 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:31.492811 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 00:20:31.492865 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:20:31.494133 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:31.487178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:38723 (local address 127.16.71.131:38835)
0811 00:20:31.488337 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:31.488389 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:31.490424 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:31.491152 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:31.491167 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:31.491302 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:31.492013 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:20:31.492016 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:31.492087 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:20:31.492220 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:31.492565 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:31.492575 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:31.492582 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:20:31.492879 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 00:20:31.493113 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":300,"threads_started":1}
I20260811 00:20:31.495622 2312 heartbeater.cc:347] Connected to a master server at 127.16.71.190:38723
I20260811 00:20:31.495994 2312 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:31.496727 2312 heartbeater.cc:511] Master 127.16.71.190:38723 requested a full tablet report, sending...
I20260811 00:20:31.498281 1519 ts_manager.cc:194] Registered new tserver with Master: e8a1d66c55ed42acb94ca132bcb7b647 (127.16.71.131:37555)
I20260811 00:20:31.505339 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:20:31.511893 1841 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:20:31.510179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.16.71.190:38723)
0811 00:20:31.510354 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:31.510358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:31.510940 (+ 582us) server_negotiation.cc:407] Connection header received
0811 00:20:31.511072 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:31.511077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:31.511136 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:31.511222 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:31.511593 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44684: BlockingRecv error: recv got EOF from 127.0.0.1:44684 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:20:31.512715 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 1609
I20260811 00:20:31.519850 2328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:31.516519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.16.71.129:34007)
0811 00:20:31.517142 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:31.517153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:31.517187 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:20:31.517266 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:31.517279 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:31.517583 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:31.517772 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:31.518923 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:31.518936 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:31.518952 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:31.518970 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:31.519025 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:31.519044 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:31.519247 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:31.519321 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 00:20:31.519428 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":291,"threads_started":1}
I20260811 00:20:31.521370 1827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:31.604197 1834 heartbeater.cc:503] Master 127.16.71.190:38723 was elected leader, sending a full tablet report...
I20260811 00:20:31.605454 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 1609
W20260811 00:20:31.634075 1497 connection.cc:535] server connection from 127.16.71.129:49379 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:20:31.634554 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 1844
I20260811 00:20:31.638398 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:31.635488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.16.71.130:37169)
0811 00:20:31.636033 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:31.636041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:31.636070 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:20:31.636149 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:31.636155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:31.636409 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:31.636574 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:31.637628 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:31.637635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:31.637647 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:31.637667 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:31.637709 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:31.637724 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:31.637856 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:31.637997 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 00:20:31.638106 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":239,"threads_started":1}
I20260811 00:20:31.639796 2064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:31.723915 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 1844
I20260811 00:20:31.750911 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 2082
I20260811 00:20:31.755366 2338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:31.752043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57670 (local address 127.16.71.131:37555)
0811 00:20:31.752794 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:31.752801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:31.752835 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:20:31.752908 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:31.752914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:31.753173 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:31.753345 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:31.754575 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:31.754582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:31.754594 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:20:31.754613 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:31.754654 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:20:31.754669 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:31.754843 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:31.754923 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 00:20:31.755050 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":379,"threads_started":1}
I20260811 00:20:31.756776 2301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:31.839516 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 2082
I20260811 00:20:31.876291 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 1460
I20260811 00:20:31.877948 1566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:31.989248 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 1460
2026-08-11T00:20:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2634 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:20:32 dist-test-slave-f7th krb5kdc[2350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:20:32 dist-test-slave-f7th krb5kdc[2350](info): set up 2 sockets
Aug 11 00:20:32 dist-test-slave-f7th krb5kdc[2350](info): commencing operation
krb5kdc: starting...
W20260811 00:20:33.307677 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:20:33 dist-test-slave-f7th krb5kdc[2350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:20:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:20:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:20:33.629519 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43173
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:43173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:34.007561 2371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:34.007927 2371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:34.008081 2371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:34.017307 2371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:20:34.017470 2371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:34.017562 2371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:34.017649 2371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:20:34.017714 2371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:20:34.028704 2371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:43173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:34.030308 2371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:34.031771 2371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:34.044930 2382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:34.045013 2384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:34.045013 2381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:34.046989 2371 server_base.cc:1034] running on GCE node
I20260811 00:20:34.048095 2371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:34.049921 2371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:34.051214 2371 hybrid_clock.cc:648] HybridClock initialized: now 1786407634051149 us; error 58 us; skew 500 ppm
I20260811 00:20:34.051668 2371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:34 dist-test-slave-f7th krb5kdc[2350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407634, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:34.059080 2371 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:20:34.061693 2371 webserver.cc:460] Webserver started at http://127.16.71.190:42261/ using document root <none> and password file <none>
I20260811 00:20:34.062518 2371 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:34.062664 2371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:34.063020 2371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:34.066475 2371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "e8931e25fa884c29b0420a7254089db8"
format_stamp: "Formatted at 2026-08-11 00:20:34 on dist-test-slave-f7th"
server_key: "d2a819abe809b75e3f521a2a8239aed2"
server_key_iv: "00b1a8f079f20bb9d41f3dfc8dbce985"
server_key_version: "encryptionkey@0"
I20260811 00:20:34.067471 2371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "e8931e25fa884c29b0420a7254089db8"
format_stamp: "Formatted at 2026-08-11 00:20:34 on dist-test-slave-f7th"
server_key: "d2a819abe809b75e3f521a2a8239aed2"
server_key_iv: "00b1a8f079f20bb9d41f3dfc8dbce985"
server_key_version: "encryptionkey@0"
I20260811 00:20:34.074733 2371 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260811 00:20:34.078814 2395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:34.080394 2371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:20:34.080603 2371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "e8931e25fa884c29b0420a7254089db8"
format_stamp: "Formatted at 2026-08-11 00:20:34 on dist-test-slave-f7th"
server_key: "d2a819abe809b75e3f521a2a8239aed2"
server_key_iv: "00b1a8f079f20bb9d41f3dfc8dbce985"
server_key_version: "encryptionkey@0"
I20260811 00:20:34.080869 2371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:34.105731 2371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:34.110471 2371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:34.110783 2371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:34.131111 2371 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:43173
I20260811 00:20:34.131151 2491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:43173 every 8 connection(s)
I20260811 00:20:34.132576 2371 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:20:34.132992 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 2371
I20260811 00:20:34.133176 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:20:34.133971 16670 external_mini_cluster.cc:1428] Setting key f8823381c2239d7415783000a81384f8
I20260811 00:20:34.137331 2493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:20:34 dist-test-slave-f7th krb5kdc[2350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:34.149570 2493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8: Bootstrap starting.
I20260811 00:20:34.153156 2493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:34.154297 2493 log.cc:824] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:34.158176 2493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8: No bootstrap required, opened a new log
I20260811 00:20:34.158756 2498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:34.136136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.16.71.190:43173)
0811 00:20:34.137118 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:34.137132 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:34.137176 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:20:34.138981 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:34.139016 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:34.139468 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:34.139886 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:34.141765 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:34.143241 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:34.144896 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:34.145672 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:34.145883 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:34.150200 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:34.150210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:34.150229 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:34.150281 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:34.153554 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:34.154392 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:34.154400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:34.154408 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:34.154627 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:34.155146 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:34.155150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:34.155153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:34.155908 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:34.156274 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:34.156756 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 00:20:34.157262 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":462,"threads_started":1}
I20260811 00:20:34.167380 2493 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8931e25fa884c29b0420a7254089db8" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43173 } }
I20260811 00:20:34.167814 2493 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:34.167896 2493 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8931e25fa884c29b0420a7254089db8, State: Initialized, Role: FOLLOWER
I20260811 00:20:34.168476 2493 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8931e25fa884c29b0420a7254089db8" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43173 } }
I20260811 00:20:34.168761 2493 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:20:34.168887 2493 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:20:34.169039 2493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:34.171389 2493 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8931e25fa884c29b0420a7254089db8" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43173 } }
I20260811 00:20:34.171952 2493 leader_election.cc:302] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8931e25fa884c29b0420a7254089db8; no voters:
I20260811 00:20:34.172693 2493 leader_election.cc:288] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:20:34.173323 2504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:34.174342 2504 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [term 1 LEADER]: Becoming Leader. State: Replica: e8931e25fa884c29b0420a7254089db8, State: Running, Role: LEADER
I20260811 00:20:34.174954 2504 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8931e25fa884c29b0420a7254089db8" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43173 } }
I20260811 00:20:34.175359 2493 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:20:34.177838 2509 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8931e25fa884c29b0420a7254089db8. Latest consensus state: current_term: 1 leader_uuid: "e8931e25fa884c29b0420a7254089db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8931e25fa884c29b0420a7254089db8" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43173 } } }
I20260811 00:20:34.178308 2509 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:34.178051 2507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8931e25fa884c29b0420a7254089db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8931e25fa884c29b0420a7254089db8" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43173 } } }
I20260811 00:20:34.178509 2507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:20:34.179337 2518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:20:34.185860 2518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:20:34.195878 2518 catalog_manager.cc:1347] Generated new cluster ID: 62282b14cbdc4ef6a1277ffa63a8c69f
I20260811 00:20:34.196062 2518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:20:34.231280 2518 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:20:34.233590 2518 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:20:34.249236 2518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8: Generated new TSK 0
I20260811 00:20:34.250371 2518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:20:34.334208 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:43173
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:34.706907 2542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:34.707269 2542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:34.707423 2542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:34.715191 2542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:34.715360 2542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:34.715525 2542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:20:34.725841 2542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:43173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:34.727352 2542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:34.728866 2542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:34.743137 2551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:34.743113 2552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:34.743335 2555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:34.744096 2542 server_base.cc:1034] running on GCE node
I20260811 00:20:34.744833 2542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:34.745909 2542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:34.747133 2542 hybrid_clock.cc:648] HybridClock initialized: now 1786407634747068 us; error 56 us; skew 500 ppm
I20260811 00:20:34.747521 2542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:34 dist-test-slave-f7th krb5kdc[2350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407634, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:34.751384 2542 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:20:34.753314 2542 webserver.cc:460] Webserver started at http://127.16.71.129:41395/ using document root <none> and password file <none>
I20260811 00:20:34.754129 2542 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:34.754256 2542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:34.754590 2542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:34.758019 2542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "0d6cb1d4450946b0977c338b04918cab"
format_stamp: "Formatted at 2026-08-11 00:20:34 on dist-test-slave-f7th"
server_key: "f417fc58f113e69ae680c4e3625b92da"
server_key_iv: "a1c1a91826d6a0bf7ededbb04278cc1f"
server_key_version: "encryptionkey@0"
I20260811 00:20:34.758894 2542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "0d6cb1d4450946b0977c338b04918cab"
format_stamp: "Formatted at 2026-08-11 00:20:34 on dist-test-slave-f7th"
server_key: "f417fc58f113e69ae680c4e3625b92da"
server_key_iv: "a1c1a91826d6a0bf7ededbb04278cc1f"
server_key_version: "encryptionkey@0"
I20260811 00:20:34.765241 2542 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:20:34.769541 2564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:34.771232 2542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:20:34.771425 2542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "0d6cb1d4450946b0977c338b04918cab"
format_stamp: "Formatted at 2026-08-11 00:20:34 on dist-test-slave-f7th"
server_key: "f417fc58f113e69ae680c4e3625b92da"
server_key_iv: "a1c1a91826d6a0bf7ededbb04278cc1f"
server_key_version: "encryptionkey@0"
I20260811 00:20:34.771656 2542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:34.813993 2542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:34.819116 2542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:34.819460 2542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:34.820681 2542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:34.822247 2542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:34.822345 2542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:34.822484 2542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:34.822556 2542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:34.857770 2542 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:40925
I20260811 00:20:34.857825 2782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:40925 every 8 connection(s)
I20260811 00:20:34.859331 2542 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:20:34.869407 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 2542
I20260811 00:20:34.869622 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:20:34.870297 16670 external_mini_cluster.cc:1428] Setting key de3dd672db39ccb0ccaaeec94871b8f0
Aug 11 00:20:34 dist-test-slave-f7th krb5kdc[2350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407634, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:34.881455 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:34.862313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:54173 (local address 127.16.71.190:43173)
0811 00:20:34.862811 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:34.862816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:34.863819 (+ 1003us) server_negotiation.cc:407] Connection header received
0811 00:20:34.865834 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:34.865841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:34.865942 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:34.866046 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:34.867806 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:34.868563 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:34.870439 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:34.870866 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:34.871005 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:34.874958 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:34.874965 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:34.874969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:34.875011 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:34.877867 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:34.878778 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:34.878784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:34.878787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:34.878938 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:34.879445 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:34.879450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:34.879452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:34.879756 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:34.879916 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:34.880768 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 00:20:34.881065 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":281,"threads_started":1}
I20260811 00:20:34.882795 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:34.862822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:43173 (local address 127.16.71.129:54173)
0811 00:20:34.863631 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:34.863688 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:34.865525 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:34.866331 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:34.866341 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:34.866949 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:34.867603 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:34.867623 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:34.868777 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:34.868781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:34.870246 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:34.870263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:34.871145 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:34.871153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:34.871425 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:34.872146 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:34.872173 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:34.874639 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:34.878066 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:34.878078 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:34.878096 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:34.878633 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:34.879059 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:34.879064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:34.879068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:34.879323 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:34.879908 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:34.879919 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:34.880308 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 00:20:34.880601 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 00:20:34.881340 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":307,"threads_started":1}
I20260811 00:20:34.885146 2786 heartbeater.cc:347] Connected to a master server at 127.16.71.190:43173
I20260811 00:20:34.885581 2786 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:34.886577 2786 heartbeater.cc:511] Master 127.16.71.190:43173 requested a full tablet report, sending...
I20260811 00:20:34.889458 2422 ts_manager.cc:194] Registered new tserver with Master: 0d6cb1d4450946b0977c338b04918cab (127.16.71.129:40925)
I20260811 00:20:34.891664 2422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:54173
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:20:34.941303 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:43173
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:20:35.309357 2795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:35.309686 2795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:35.309785 2795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:35.317425 2795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:35.317566 2795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:35.317683 2795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:20:35.328176 2795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:43173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:35.329711 2795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:35.331257 2795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:35.345990 2806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:35.345957 2810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:35.346024 2807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:35.347039 2795 server_base.cc:1034] running on GCE node
I20260811 00:20:35.347736 2795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:35.348850 2795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:35.350103 2795 hybrid_clock.cc:648] HybridClock initialized: now 1786407635350046 us; error 47 us; skew 500 ppm
I20260811 00:20:35.350436 2795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:35 dist-test-slave-f7th krb5kdc[2350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407635, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:35.353808 2795 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:20:35.355592 2795 webserver.cc:460] Webserver started at http://127.16.71.130:41753/ using document root <none> and password file <none>
I20260811 00:20:35.356367 2795 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:35.356501 2795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:35.356941 2795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:35.360423 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "0d93ff56fc80467884003ee28146b2e3"
format_stamp: "Formatted at 2026-08-11 00:20:35 on dist-test-slave-f7th"
server_key: "f3ecadc05b32cb8e34467fda2cdad872"
server_key_iv: "afbe880400c6069e9640c640f3a4a788"
server_key_version: "encryptionkey@0"
I20260811 00:20:35.361522 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "0d93ff56fc80467884003ee28146b2e3"
format_stamp: "Formatted at 2026-08-11 00:20:35 on dist-test-slave-f7th"
server_key: "f3ecadc05b32cb8e34467fda2cdad872"
server_key_iv: "afbe880400c6069e9640c640f3a4a788"
server_key_version: "encryptionkey@0"
I20260811 00:20:35.367782 2795 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:20:35.371796 2824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:35.373548 2795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:20:35.373750 2795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "0d93ff56fc80467884003ee28146b2e3"
format_stamp: "Formatted at 2026-08-11 00:20:35 on dist-test-slave-f7th"
server_key: "f3ecadc05b32cb8e34467fda2cdad872"
server_key_iv: "afbe880400c6069e9640c640f3a4a788"
server_key_version: "encryptionkey@0"
I20260811 00:20:35.373965 2795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:35.407490 2795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:35.412561 2795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:35.412919 2795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:35.414014 2795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:35.415565 2795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:35.415653 2795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:35.415794 2795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:35.415858 2795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:35.450834 2795 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:39389
I20260811 00:20:35.450875 3012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:39389 every 8 connection(s)
I20260811 00:20:35.452365 2795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:20:35.453343 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 2795
I20260811 00:20:35.453552 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:20:35.454167 16670 external_mini_cluster.cc:1428] Setting key d9c687ea7118e1a41e6c55f006f0f258
Aug 11 00:20:35 dist-test-slave-f7th krb5kdc[2350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407635, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:35.477823 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:35.455810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:48813 (local address 127.16.71.190:43173)
0811 00:20:35.456427 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:35.456433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:35.458288 (+ 1855us) server_negotiation.cc:407] Connection header received
0811 00:20:35.460511 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:35.460518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:35.460650 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:35.460777 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:35.462830 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:35.463894 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:35.465927 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:35.466558 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:35.466711 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:35.471003 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:35.471009 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:35.471013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:35.471056 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:35.473917 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:35.474840 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:35.474845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:35.474849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:35.474998 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:35.475620 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:35.475624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:35.475626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:35.475980 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:35.476128 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:35.477149 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0811 00:20:35.477433 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":379,"threads_started":1}
I20260811 00:20:35.478726 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:35.456888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:43173 (local address 127.16.71.130:48813)
0811 00:20:35.457948 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:35.458062 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:35.460111 (+ 2049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:35.461128 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:35.461138 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:35.461783 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:35.462566 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:35.462592 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:35.464171 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:35.464176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:35.465695 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:35.465719 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:35.466907 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:35.466915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:35.467191 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:35.467997 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:35.468033 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:35.470678 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:35.474124 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:35.474135 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:35.474153 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:35.474679 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:35.475148 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:35.475154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:35.475158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:35.475399 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:35.476118 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:35.476127 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:35.476550 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 00:20:35.476965 (+ 415us) client_negotiation.cc:240] Negotiation successful
0811 00:20:35.477430 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":422,"threads_started":1}
I20260811 00:20:35.480670 3014 heartbeater.cc:347] Connected to a master server at 127.16.71.190:43173
I20260811 00:20:35.481092 3014 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:35.482122 3014 heartbeater.cc:511] Master 127.16.71.190:43173 requested a full tablet report, sending...
I20260811 00:20:35.484329 2422 ts_manager.cc:194] Registered new tserver with Master: 0d93ff56fc80467884003ee28146b2e3 (127.16.71.130:39389)
I20260811 00:20:35.485324 2422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:48813
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:20:35.524812 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:43173
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:20:35.895779 2786 heartbeater.cc:503] Master 127.16.71.190:43173 was elected leader, sending a full tablet report...
W20260811 00:20:35.899214 3023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:20:35.899529 3023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:20:35.899628 3023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:20:35.907397 3023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:20:35.907543 3023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:20:35.907701 3023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:20:35.918325 3023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:43173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:20:35.919839 3023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:20:35.921275 3023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:20:35.935340 3034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:35.935585 3035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:20:35.935604 3039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:20:35.936308 3023 server_base.cc:1034] running on GCE node
I20260811 00:20:35.937080 3023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:20:35.938225 3023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:20:35.939502 3023 hybrid_clock.cc:648] HybridClock initialized: now 1786407635939445 us; error 52 us; skew 500 ppm
I20260811 00:20:35.939956 3023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:20:35 dist-test-slave-f7th krb5kdc[2350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407635, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:20:35.943737 3023 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:20:35.945545 3023 webserver.cc:460] Webserver started at http://127.16.71.131:34529/ using document root <none> and password file <none>
I20260811 00:20:35.946326 3023 fs_manager.cc:359] Metadata directory not provided
I20260811 00:20:35.946444 3023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:20:35.946720 3023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:20:35.950093 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "d560b6f6c6d2436f912d944a2c2b8677"
format_stamp: "Formatted at 2026-08-11 00:20:35 on dist-test-slave-f7th"
server_key: "0bb82db18c57ca310218493827d64d98"
server_key_iv: "b10f27a49feee090df16087e441f4de8"
server_key_version: "encryptionkey@0"
I20260811 00:20:35.950920 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "d560b6f6c6d2436f912d944a2c2b8677"
format_stamp: "Formatted at 2026-08-11 00:20:35 on dist-test-slave-f7th"
server_key: "0bb82db18c57ca310218493827d64d98"
server_key_iv: "b10f27a49feee090df16087e441f4de8"
server_key_version: "encryptionkey@0"
I20260811 00:20:35.957115 3023 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:20:35.961469 3049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:35.963267 3023 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:20:35.963455 3023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "d560b6f6c6d2436f912d944a2c2b8677"
format_stamp: "Formatted at 2026-08-11 00:20:35 on dist-test-slave-f7th"
server_key: "0bb82db18c57ca310218493827d64d98"
server_key_iv: "b10f27a49feee090df16087e441f4de8"
server_key_version: "encryptionkey@0"
I20260811 00:20:35.963666 3023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:20:35.979795 3023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:20:35.984512 3023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:20:35.984865 3023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:20:35.986028 3023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:20:35.987650 3023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:20:35.987744 3023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:35.987839 3023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:20:35.987888 3023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:20:36.023155 3023 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:40915
I20260811 00:20:36.023204 3264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:40915 every 8 connection(s)
I20260811 00:20:36.024524 3023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:20:36.028061 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 3023
I20260811 00:20:36.028278 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:20:36.028885 16670 external_mini_cluster.cc:1428] Setting key 2192079ba67de01b283263120dfc67b2
Aug 11 00:20:36 dist-test-slave-f7th krb5kdc[2350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407635, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:20:36.047807 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.027195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:35477 (local address 127.16.71.190:43173)
0811 00:20:36.027676 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.027682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.028893 (+ 1211us) server_negotiation.cc:407] Connection header received
0811 00:20:36.031342 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.031346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.031451 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.031568 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:36.033677 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.034552 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.036395 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.036943 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.037057 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.041219 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:36.041225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:36.041229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:36.041282 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:36.044379 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.045402 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.045409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.045412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.045593 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.046120 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.046123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.046125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.046414 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:36.046541 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.047297 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.047489 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":243,"threads_started":1}
I20260811 00:20:36.049110 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.027709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:43173 (local address 127.16.71.131:35477)
0811 00:20:36.028681 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:36.028738 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:36.031003 (+ 2265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:36.031848 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:36.031858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:36.032547 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:20:36.033450 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.033471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.034755 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.034760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.036192 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.036210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.037209 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.037215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.037460 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.038255 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:20:36.038282 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:20:36.040938 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:20:36.044564 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:36.044577 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:36.044597 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:36.045253 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:36.045721 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:20:36.045725 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:20:36.045729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:20:36.045968 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:20:36.046514 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:20:36.046521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:20:36.046949 (+ 428us) client_negotiation.cc:769] Sending connection context
0811 00:20:36.047192 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 00:20:36.047632 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":329,"threads_started":1}
I20260811 00:20:36.050668 3265 heartbeater.cc:347] Connected to a master server at 127.16.71.190:43173
I20260811 00:20:36.051057 3265 heartbeater.cc:464] Registering TS with master...
I20260811 00:20:36.051872 3265 heartbeater.cc:511] Master 127.16.71.190:43173 requested a full tablet report, sending...
I20260811 00:20:36.053573 2422 ts_manager.cc:194] Registered new tserver with Master: d560b6f6c6d2436f912d944a2c2b8677 (127.16.71.131:40915)
I20260811 00:20:36.054418 2422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:35477
I20260811 00:20:36.058285 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:20:36.080109 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.063207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55890 (local address 127.16.71.190:43173)
0811 00:20:36.063431 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.063438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.063823 (+ 385us) server_negotiation.cc:407] Connection header received
0811 00:20:36.064072 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.064076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.064163 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.064253 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:36.065861 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.066632 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.068034 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.068510 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.068676 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.071622 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:36.071628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:36.071631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:36.071672 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:36.074948 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.076224 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.076231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.076235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.076507 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.077120 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.077126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.077129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.077738 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:36.078021 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.079499 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.079797 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:20:36.087092 2422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:20:36.094765 2422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:20:36.137720 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.108317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:39389 (local address 127.0.0.1:52416)
0811 00:20:36.125121 (+ 16804us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:36.125146 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:36.125293 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:36.126164 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:36.126172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:36.126225 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:36.126751 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.126764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.129129 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.129135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.132418 (+ 3283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.132439 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.136470 (+ 4031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.136477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.136747 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.136812 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 00:20:36.136977 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 00:20:36.137212 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16649,"mutex_wait_us":3,"thread_start_us":15592,"threads_started":1}
I20260811 00:20:36.139292 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.108982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:40915 (local address 127.0.0.1:56324)
0811 00:20:36.129161 (+ 20179us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:36.129190 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:36.129372 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:36.130533 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:36.130538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:36.130566 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:36.131118 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.131133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.133261 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.133267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.135882 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.135905 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.138339 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.138347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.138554 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.138589 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:20:36.138687 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 00:20:36.138846 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20036,"mutex_wait_us":75,"thread_start_us":19844,"threads_started":1}
I20260811 00:20:36.139269 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.108420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.16.71.130:39389)
0811 00:20:36.124754 (+ 16334us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.124764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.125434 (+ 670us) server_negotiation.cc:407] Connection header received
0811 00:20:36.125567 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.125576 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.125873 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.127080 (+ 1207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:36.127359 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.128941 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.132685 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.135917 (+ 3232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.136064 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.136304 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.138734 (+ 2430us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.138955 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16135,"thread_start_us":15993,"threads_started":1}
I20260811 00:20:36.140331 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.108996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.16.71.131:40915)
0811 00:20:36.127895 (+ 18899us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.127906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.129426 (+ 1520us) server_negotiation.cc:407] Connection header received
0811 00:20:36.129556 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.129563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.129907 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.130198 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:36.131359 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.133078 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.136133 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.137735 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.137880 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.138158 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.139770 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.139996 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18650,"thread_start_us":20118,"threads_started":1}
I20260811 00:20:36.140651 3296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.108284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:40925 (local address 127.0.0.1:49588)
0811 00:20:36.123335 (+ 15051us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:36.123408 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:36.123665 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:36.130192 (+ 6527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:36.130197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:36.130235 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:36.130775 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.130789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.133261 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.133268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.135998 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.136015 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.139736 (+ 3721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.139744 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.139936 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.139972 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:20:36.140049 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 00:20:36.140211 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14657,"mutex_wait_us":133,"thread_start_us":16487,"threads_started":1}
I20260811 00:20:36.142298 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.108917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.16.71.129:40925)
0811 00:20:36.128523 (+ 19606us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.128533 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.128574 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:20:36.129610 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.129618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.129910 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.131019 (+ 1109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:36.131292 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.133078 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.136230 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.139457 (+ 3227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.141005 (+ 1548us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.141278 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.141723 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.141930 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19390,"thread_start_us":21739,"threads_started":1}
I20260811 00:20:36.144348 2909 tablet_service.cc:1467] Processing CreateTablet for tablet befc1d5de2a34eb5b2e7f83807de741e (DEFAULT_TABLE table=test-table [id=00d2f902c6bc46f2a722c62645b2f9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:36.144783 3142 tablet_service.cc:1467] Processing CreateTablet for tablet befc1d5de2a34eb5b2e7f83807de741e (DEFAULT_TABLE table=test-table [id=00d2f902c6bc46f2a722c62645b2f9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:36.146178 3142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befc1d5de2a34eb5b2e7f83807de741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:36.146180 2909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befc1d5de2a34eb5b2e7f83807de741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:36.147585 2656 tablet_service.cc:1467] Processing CreateTablet for tablet befc1d5de2a34eb5b2e7f83807de741e (DEFAULT_TABLE table=test-table [id=00d2f902c6bc46f2a722c62645b2f9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:20:36.149470 2656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befc1d5de2a34eb5b2e7f83807de741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:20:36.163199 3302 tablet_bootstrap.cc:492] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Bootstrap starting.
I20260811 00:20:36.164353 3303 tablet_bootstrap.cc:492] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Bootstrap starting.
I20260811 00:20:36.168530 3303 tablet_bootstrap.cc:654] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:36.169106 3302 tablet_bootstrap.cc:654] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:36.170060 3303 log.cc:824] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:36.170377 3302 log.cc:824] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:36.171547 3304 tablet_bootstrap.cc:492] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Bootstrap starting.
I20260811 00:20:36.174710 3302 tablet_bootstrap.cc:492] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: No bootstrap required, opened a new log
I20260811 00:20:36.174919 3303 tablet_bootstrap.cc:492] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: No bootstrap required, opened a new log
I20260811 00:20:36.175017 3302 ts_tablet_manager.cc:1397] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 00:20:36.175208 3303 ts_tablet_manager.cc:1397] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260811 00:20:36.175626 3304 tablet_bootstrap.cc:654] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Neither blocks nor log segments found. Creating new log.
I20260811 00:20:36.177071 3304 log.cc:824] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Log is configured to *not* fsync() on all Append() calls
I20260811 00:20:36.183660 3302 raft_consensus.cc:348] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.184078 3302 raft_consensus.cc:374] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:36.184263 3302 raft_consensus.cc:729] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6cb1d4450946b0977c338b04918cab, State: Initialized, Role: FOLLOWER
I20260811 00:20:36.185012 3302 consensus_queue.cc:260] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.186877 3302 ts_tablet_manager.cc:1428] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Time spent starting tablet: real 0.012s user 0.002s sys 0.010s
I20260811 00:20:36.186933 3304 tablet_bootstrap.cc:492] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: No bootstrap required, opened a new log
I20260811 00:20:36.187252 3304 ts_tablet_manager.cc:1397] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260811 00:20:36.189245 3303 raft_consensus.cc:348] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.189755 3303 raft_consensus.cc:374] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:36.189857 3303 raft_consensus.cc:729] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d93ff56fc80467884003ee28146b2e3, State: Initialized, Role: FOLLOWER
I20260811 00:20:36.190507 3303 consensus_queue.cc:260] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.191732 3014 heartbeater.cc:503] Master 127.16.71.190:43173 was elected leader, sending a full tablet report...
I20260811 00:20:36.193874 3303 ts_tablet_manager.cc:1428] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
I20260811 00:20:36.196973 3304 raft_consensus.cc:348] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.197374 3304 raft_consensus.cc:374] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:20:36.197458 3304 raft_consensus.cc:729] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d560b6f6c6d2436f912d944a2c2b8677, State: Initialized, Role: FOLLOWER
I20260811 00:20:36.197999 3304 consensus_queue.cc:260] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.199067 3265 heartbeater.cc:503] Master 127.16.71.190:43173 was elected leader, sending a full tablet report...
I20260811 00:20:36.199566 3304 ts_tablet_manager.cc:1428] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Time spent starting tablet: real 0.012s user 0.009s sys 0.005s
W20260811 00:20:36.205072 3015 tablet.cc:2369] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:20:36.205781 3320 raft_consensus.cc:482] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:20:36.206086 3320 raft_consensus.cc:504] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.207824 3320 leader_election.cc:288] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d6cb1d4450946b0977c338b04918cab (127.16.71.129:40925), d560b6f6c6d2436f912d944a2c2b8677 (127.16.71.131:40915)
I20260811 00:20:36.214687 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.208575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:50097 (local address 127.16.71.131:40915)
0811 00:20:36.208834 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.208840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.209299 (+ 459us) server_negotiation.cc:407] Connection header received
0811 00:20:36.209490 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.209494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.209558 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.209633 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:36.210202 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.210943 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.212962 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.213760 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.213926 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.214013 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.214305 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.214442 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 00:20:36.214788 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.208235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:40915 (local address 127.16.71.130:50097)
0811 00:20:36.209158 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:36.209180 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:36.209347 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:36.209678 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:36.209682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:36.209708 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:36.210048 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.210058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.211094 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.211098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.212788 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.212805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.213995 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.214002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.214143 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.214168 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 00:20:36.214258 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 00:20:36.214406 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":527,"threads_started":1}
I20260811 00:20:36.214911 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.208307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:60495 (local address 127.16.71.129:40925)
0811 00:20:36.208568 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.208575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.209514 (+ 939us) server_negotiation.cc:407] Connection header received
0811 00:20:36.209693 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.209699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.209765 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.209846 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:20:36.210453 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.211181 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.213148 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.213977 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.214137 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.214214 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.214521 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.214660 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:20:36.215476 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.208241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:40925 (local address 127.16.71.130:60495)
0811 00:20:36.209387 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0811 00:20:36.209402 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:20:36.209547 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:20:36.209888 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:20:36.209892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:20:36.209917 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:20:36.210300 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.210309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.211326 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.211330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.212980 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:20:36.212995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.214198 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.214204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:20:36.214360 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.214384 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:20:36.214471 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:20:36.214608 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042,"mutex_wait_us":51,"thread_start_us":735,"threads_started":1}
I20260811 00:20:36.215866 3191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befc1d5de2a34eb5b2e7f83807de741e" candidate_uuid: "0d93ff56fc80467884003ee28146b2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d560b6f6c6d2436f912d944a2c2b8677" is_pre_election: true
I20260811 00:20:36.216495 3191 raft_consensus.cc:2454] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d93ff56fc80467884003ee28146b2e3 in term 0.
I20260811 00:20:36.216575 2698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befc1d5de2a34eb5b2e7f83807de741e" candidate_uuid: "0d93ff56fc80467884003ee28146b2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6cb1d4450946b0977c338b04918cab" is_pre_election: true
I20260811 00:20:36.217173 2698 raft_consensus.cc:2454] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d93ff56fc80467884003ee28146b2e3 in term 0.
I20260811 00:20:36.217235 2825 leader_election.cc:302] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d93ff56fc80467884003ee28146b2e3, d560b6f6c6d2436f912d944a2c2b8677; no voters:
I20260811 00:20:36.217733 3320 raft_consensus.cc:2781] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:20:36.217895 3320 raft_consensus.cc:482] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:20:36.218010 3320 raft_consensus.cc:3037] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:36.220391 3320 raft_consensus.cc:504] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.221069 3320 leader_election.cc:288] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [CANDIDATE]: Term 1 election: Requested vote from peers 0d6cb1d4450946b0977c338b04918cab (127.16.71.129:40925), d560b6f6c6d2436f912d944a2c2b8677 (127.16.71.131:40915)
I20260811 00:20:36.221659 2698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befc1d5de2a34eb5b2e7f83807de741e" candidate_uuid: "0d93ff56fc80467884003ee28146b2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6cb1d4450946b0977c338b04918cab"
I20260811 00:20:36.221706 3191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befc1d5de2a34eb5b2e7f83807de741e" candidate_uuid: "0d93ff56fc80467884003ee28146b2e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d560b6f6c6d2436f912d944a2c2b8677"
I20260811 00:20:36.221863 2698 raft_consensus.cc:3037] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:36.221962 3191 raft_consensus.cc:3037] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:20:36.224155 2698 raft_consensus.cc:2454] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d93ff56fc80467884003ee28146b2e3 in term 1.
I20260811 00:20:36.224254 3191 raft_consensus.cc:2454] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d93ff56fc80467884003ee28146b2e3 in term 1.
I20260811 00:20:36.224799 2825 leader_election.cc:302] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d93ff56fc80467884003ee28146b2e3, d560b6f6c6d2436f912d944a2c2b8677; no voters:
I20260811 00:20:36.225143 3320 raft_consensus.cc:2781] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:20:36.225615 3320 raft_consensus.cc:686] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 1 LEADER]: Becoming Leader. State: Replica: 0d93ff56fc80467884003ee28146b2e3, State: Running, Role: LEADER
I20260811 00:20:36.226197 3320 consensus_queue.cc:237] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } }
I20260811 00:20:36.231115 2422 catalog_manager.cc:5495] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d93ff56fc80467884003ee28146b2e3 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "0d93ff56fc80467884003ee28146b2e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d93ff56fc80467884003ee28146b2e3" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 39389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 } health_report { overall_health: UNKNOWN } } }
I20260811 00:20:36.253302 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.246930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.16.71.130:39389)
0811 00:20:36.247197 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.247202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.247223 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:20:36.247412 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.247417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.247522 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.247669 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:20:36.248169 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.249078 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.250626 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.251091 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.251204 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.251876 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:20:36.252468 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:20:36.252595 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.252863 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.252957 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 00:20:36.263479 3191 raft_consensus.cc:1261] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 1 FOLLOWER]: Refusing update from remote peer 0d93ff56fc80467884003ee28146b2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:36.263479 2698 raft_consensus.cc:1261] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 1 FOLLOWER]: Refusing update from remote peer 0d93ff56fc80467884003ee28146b2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:20:36.264736 3320 consensus_queue.cc:1035] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d560b6f6c6d2436f912d944a2c2b8677" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 40915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:20:36.265093 3328 consensus_queue.cc:1035] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d6cb1d4450946b0977c338b04918cab" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:20:36.276197 3268 tablet.cc:2369] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:20:36.282317 3335 mvcc.cc:205] Tried to move back new op lower bound from 7317125678121906176 to 7317125677986787328. Current Snapshot: MvccSnapshot[applied={T|T < 7317125678121906176}]
I20260811 00:20:36.303905 2422 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:36.304239 2422 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:20:36.308904 2422 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8931e25fa884c29b0420a7254089db8: Sending DeleteTablet for 3 replicas of tablet befc1d5de2a34eb5b2e7f83807de741e
I20260811 00:20:36.310326 2656 tablet_service.cc:1514] Processing DeleteTablet for tablet befc1d5de2a34eb5b2e7f83807de741e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:36 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:49588
I20260811 00:20:36.310309 2909 tablet_service.cc:1514] Processing DeleteTablet for tablet befc1d5de2a34eb5b2e7f83807de741e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:36 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:52416
I20260811 00:20:36.310462 3142 tablet_service.cc:1514] Processing DeleteTablet for tablet befc1d5de2a34eb5b2e7f83807de741e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:20:36 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:56324
I20260811 00:20:36.313242 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 2542
I20260811 00:20:36.321894 3360 tablet_replica.cc:330] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: stopping tablet replica
I20260811 00:20:36.322696 3360 raft_consensus.cc:2229] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:36.323520 3360 raft_consensus.cc:2258] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:36.326361 3360 ts_tablet_manager.cc:1905] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:36.331476 3358 tablet_replica.cc:330] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: stopping tablet replica
I20260811 00:20:36.331708 3361 tablet_replica.cc:330] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: stopping tablet replica
I20260811 00:20:36.332270 3361 raft_consensus.cc:2229] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:20:36.332310 3358 raft_consensus.cc:2229] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:20:36.332973 3361 raft_consensus.cc:2258] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:36.333325 3358 raft_consensus.cc:2258] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:20:36.335752 3361 ts_tablet_manager.cc:1905] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:36.335758 3358 ts_tablet_manager.cc:1905] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:20:36.338711 3360 ts_tablet_manager.cc:1918] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:36.338953 3360 log.cc:1196] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/befc1d5de2a34eb5b2e7f83807de741e
I20260811 00:20:36.339701 3360 ts_tablet_manager.cc:1939] T befc1d5de2a34eb5b2e7f83807de741e P 0d6cb1d4450946b0977c338b04918cab: Deleting consensus metadata
I20260811 00:20:36.342291 2403 catalog_manager.cc:4841] TS 0d6cb1d4450946b0977c338b04918cab (127.16.71.129:40925): tablet befc1d5de2a34eb5b2e7f83807de741e (table test-table [id=00d2f902c6bc46f2a722c62645b2f9da]) successfully deleted
I20260811 00:20:36.344920 3358 ts_tablet_manager.cc:1918] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:36.345304 3358 log.cc:1196] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/befc1d5de2a34eb5b2e7f83807de741e
I20260811 00:20:36.346058 3358 ts_tablet_manager.cc:1939] T befc1d5de2a34eb5b2e7f83807de741e P 0d93ff56fc80467884003ee28146b2e3: Deleting consensus metadata
I20260811 00:20:36.347882 3361 ts_tablet_manager.cc:1918] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:20:36.348073 3361 log.cc:1196] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/befc1d5de2a34eb5b2e7f83807de741e
I20260811 00:20:36.348402 2402 catalog_manager.cc:4841] TS 0d93ff56fc80467884003ee28146b2e3 (127.16.71.130:39389): tablet befc1d5de2a34eb5b2e7f83807de741e (table test-table [id=00d2f902c6bc46f2a722c62645b2f9da]) successfully deleted
I20260811 00:20:36.348763 3361 ts_tablet_manager.cc:1939] T befc1d5de2a34eb5b2e7f83807de741e P d560b6f6c6d2436f912d944a2c2b8677: Deleting consensus metadata
I20260811 00:20:36.350471 2399 catalog_manager.cc:4841] TS d560b6f6c6d2436f912d944a2c2b8677 (127.16.71.131:40915): tablet befc1d5de2a34eb5b2e7f83807de741e (table test-table [id=00d2f902c6bc46f2a722c62645b2f9da]) successfully deleted
Aug 11 00:20:36 dist-test-slave-f7th krb5kdc[2350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:20:36.359165 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.315204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.16.71.129:40925)
0811 00:20:36.315448 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.315455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.333142 (+ 17687us) server_negotiation.cc:407] Connection header received
0811 00:20:36.338207 (+ 5065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.338215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.338337 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.338420 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:36.340989 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.342125 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.344092 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.344742 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.344860 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.352535 (+ 7675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:36.352558 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:36.352575 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:36.352664 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:36.355894 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.356844 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.356853 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.356855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.357025 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.357436 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.357440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.357442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.357869 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:36.358195 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.358550 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.358800 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:20:36.359864 2775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:36.473948 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 2542
I20260811 00:20:36.497403 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 2795
Aug 11 00:20:36 dist-test-slave-f7th krb5kdc[2350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:20:36.517089 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.498383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.16.71.130:39389)
0811 00:20:36.498600 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.498606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.498628 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:20:36.498709 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.498715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.498792 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.498916 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:36.500156 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.501385 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.502725 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.503443 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.503596 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.508818 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:36.508844 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:36.508877 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:36.508943 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:36.513425 (+ 4482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.514401 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.514412 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.514416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.514628 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.515115 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.515121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.515124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.515684 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:36.516097 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.516363 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.516669 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:20:36.518007 3005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:36.636806 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 2795
I20260811 00:20:36.666694 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 3023
Aug 11 00:20:36 dist-test-slave-f7th krb5kdc[2350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:20:36.683620 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:20:36.667660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.16.71.131:40915)
0811 00:20:36.667858 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:20:36.667864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:20:36.667889 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:20:36.668051 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:20:36.668055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:20:36.668126 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:20:36.668212 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:20:36.669686 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.670439 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.671739 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:20:36.672153 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:20:36.672275 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:20:36.677237 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:20:36.677256 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:20:36.677276 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:20:36.677333 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:20:36.680349 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.681331 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.681340 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.681343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.681569 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:20:36.682013 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:20:36.682017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:20:36.682019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:20:36.682450 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:20:36.682713 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 00:20:36.683129 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 00:20:36.683343 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:20:36.684358 3260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:36.776014 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 3023
I20260811 00:20:36.798765 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 2371
I20260811 00:20:36.800077 2480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:20:36.914153 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 2371
2026-08-11T00:20:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4923 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:20:37 dist-test-slave-f7th krb5kdc[3374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:20:37 dist-test-slave-f7th krb5kdc[3374](info): set up 2 sockets
Aug 11 00:20:37 dist-test-slave-f7th krb5kdc[3374](info): commencing operation
krb5kdc: starting...
W20260811 00:20:38.236601 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.005s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:20:38 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:20:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:20:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/rangeradmin_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/rangeradmin_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/rangerlookup_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/rangerlookup_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
I20260811 00:20:38.688117 16670 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9TQxI2/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/postgres -l logfile start
2026-08-11 00:20:41.543 UTC [3422] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 00:20:41.543 UTC [3422] LOG: listening on IPv4 address "127.16.71.148", port 36395
2026-08-11 00:20:41.547 UTC [3422] LOG: listening on Unix socket "/tmp/.s.PGSQL.36395"
2026-08-11 00:20:41.572 UTC [3427] LOG: database system was shut down at 2026-08-11 00:20:39 UTC
2026-08-11 00:20:41.576 UTC [3422] LOG: database system is ready to accept connections
I20260811 00:20:42.733623 16670 mini_postgres.cc:96] Postgres bound to 36395
2026-08-11 00:20:42.759 UTC [3444] FATAL: database "slave" does not exist
127.16.71.148:36395 - accepting connections
I20260811 00:20:42.759943 16670 mini_ranger.cc:161] Starting Ranger...
I20260811 00:20:42.796334 16670 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 00:20:43.391037 16670 mini_ranger.cc:87] Created ranger Postgres database
I20260811 00:20:43.391237 16670 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-admin
2026-08-11 00:20:43,769 [I] DB FLAVOR :POSTGRES
2026-08-11 00:20:43,769 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:20:43,769 [I] Checking connection..
2026-08-11 00:20:43,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 00:20:44,058 [I] Checking connection passed.
2026-08-11 00:20:44,059 [I] --------- Verifying version history table ---------
2026-08-11 00:20:44,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:20:44,332 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:20:44,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:20:44,605 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:20:44,606 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 00:20:44,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 00:20:44.880 UTC [3562] WARNING: there is no transaction in progress
2026-08-11 00:20:44,888 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 00:20:44,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:20:45,154 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 00:20:45,154 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:20:45,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 00:20:45,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 00:20:45,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-11 00:20:45,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:20:46,276 [I] Table x_portal_user does not exist in database ranger
2026-08-11 00:20:46,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:20:46,546 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 00:20:46,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:20:46,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:20:47,112 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 00:20:47,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 00:20:48.249 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.269 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.288 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.306 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.326 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.346 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.408 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.417 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.435 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.449 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.463 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.481 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.492 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.501 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.509 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:48.518 UTC [3766] WARNING: there is no transaction in progress
2026-08-11 00:20:49,341 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 00:20:49,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 00:20:49,620 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 00:20:49,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:20:49,874 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 00:20:49,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:20:50,133 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 00:20:50,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:20:50,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:20:50,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-11 00:20:50,944 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 00:20:50,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 00:20:51,203 [I] DB_PATCHES have already been applied
I20260811 00:20:51.209684 16670 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-admin:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9TQxI2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 00:20:51.209830 16670 mini_ranger.cc:193] Using host: 127.16.71.148
I20260811 00:20:51.232115 16670 mini_ranger.cc:227] Ranger admin URL: http://127.16.71.148:50689
Aug 11, 2026 12:20:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:20:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:20:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:20:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:20:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:20:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:20:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 00:20:53.997153 16670 mini_ranger.cc:160] Time spent starting Ranger: real 11.237s user 0.010s sys 0.077s
Aug 11, 2026 12:21:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:21:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 12:21:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:21:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:21:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 00:21:17.551676 16670 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/rangerkms_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/rangerkms_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.71.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.71.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 00:21:17 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407677, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 00:21:17.817696 16670 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 00:21:17.855324 16670 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 00:21:18.912482 16670 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 00:21:18.912678 16670 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms
I20260811 00:21:18.912838 16670 mini_ranger_kms.cc:227] Using postgres at 127.16.71.148:36395
2026-08-11 00:21:19,679 [I] DB FLAVOR :POSTGRES
2026-08-11 00:21:19,679 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:21:19,679 [I] Checking connection
2026-08-11 00:21:19,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 00:21:19,960 [I] connection success
2026-08-11 00:21:19,960 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 00:21:19,960 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 00:21:19,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 00:21:20,212 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 00:21:20,213 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:21:20,213 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 00:21:20,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:36395/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 00:21:20,509 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 00:21:21.049630 16670 mini_ranger_kms.cc:310] Created kms service
I20260811 00:21:21.303435 16670 mini_ranger_kms.cc:326] Created kudu user
I20260811 00:21:21.357990 16670 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 00:21:21 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407677, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:21:21.816721 16670 mini_ranger_kms.cc:384] Added ranger policy
I20260811 00:21:21.818729 16670 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms:/tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 00:21:21.818820 16670 mini_ranger_kms.cc:240] Using host: 127.16.71.148
I20260811 00:21:21.847975 16670 mini_ranger_kms.cc:276] Ranger KMS PID: 4112
I20260811 00:21:21.848165 16670 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.71.148:40111
Aug 11, 2026 12:21:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:21:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:21:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:21:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:21:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:21:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 00:21:24 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407684, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.71.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:21:24.546775 16670 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.729s user 0.124s sys 0.477s
I20260811 00:21:24.547137 16670 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 00:21:24.547348 16670 mini_ranger_kms.cc:401] 127.16.71.148:40111/kms/v1/keys
Aug 11 00:21:24 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407684, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.71.148@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
Aug 11 00:21:25 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407684, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.71.148@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
Aug 11 00:21:27 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:21:27.717449 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40705
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:40705
--ranger_config_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:40111/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:21:28.108100 4195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:21:28.108426 4195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:21:28.108543 4195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:21:28.116338 4195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:21:28.116487 4195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:21:28.116535 4195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:21:28.116647 4195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:21:28.116719 4195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:21:28.127419 4195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.71.148:40111/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:40705
--ranger_config_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:21:28.128870 4195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:21:28.130295 4195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:21:28.143450 4205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:21:28.143610 4207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:21:28.143569 4210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:21:28.146458 4195 server_base.cc:1034] running on GCE node
I20260811 00:21:28.147485 4195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:21:28.149262 4195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:21:28.150516 4195 hybrid_clock.cc:648] HybridClock initialized: now 1786407688150473 us; error 44 us; skew 500 ppm
I20260811 00:21:28.150938 4195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:21:28 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:21:28.158664 4195 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:21:28.160684 4195 webserver.cc:460] Webserver started at http://127.16.71.190:39569/ using document root <none> and password file <none>
I20260811 00:21:28.161480 4195 fs_manager.cc:359] Metadata directory not provided
I20260811 00:21:28.161604 4195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:21:28.161931 4195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:21:28 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:21:28.225566 4195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "5c194119d5fd4040acaec77eefcecd4f"
format_stamp: "Formatted at 2026-08-11 00:21:28 on dist-test-slave-f7th"
server_key: "b7fdb29d40031aebc6988326f274197b"
server_key_iv: "01aad69d2e7e8c93e44d8d4e29f7b105"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:28.226806 4195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "5c194119d5fd4040acaec77eefcecd4f"
format_stamp: "Formatted at 2026-08-11 00:21:28 on dist-test-slave-f7th"
server_key: "b7fdb29d40031aebc6988326f274197b"
server_key_iv: "01aad69d2e7e8c93e44d8d4e29f7b105"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:28.235354 4195 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260811 00:21:28.258711 4220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:28.260814 4195 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260811 00:21:28.261076 4195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "5c194119d5fd4040acaec77eefcecd4f"
format_stamp: "Formatted at 2026-08-11 00:21:28 on dist-test-slave-f7th"
server_key: "b7fdb29d40031aebc6988326f274197b"
server_key_iv: "01aad69d2e7e8c93e44d8d4e29f7b105"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:28.261394 4195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:21:28.311450 4195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:21:28.315758 4195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:21:28.316053 4195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:21:28.335852 4195 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:40705
I20260811 00:21:28.335887 4318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:40705 every 8 connection(s)
I20260811 00:21:28.337375 4195 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:21:28.337809 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 4195
I20260811 00:21:28.338053 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:21:28.338920 16670 external_mini_cluster.cc:1428] Setting key 9dd798b76a2930c1ecb2a90cd85e3351
I20260811 00:21:28.342568 4319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:21:28 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:21:28.356177 4319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f: Bootstrap starting.
I20260811 00:21:28.359704 4319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f: Neither blocks nor log segments found. Creating new log.
I20260811 00:21:28.361171 4319 log.cc:824] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:21:28.363554 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:28.341246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.16.71.190:40705)
0811 00:21:28.342105 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:28.342117 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:28.342160 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:21:28.343890 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:28.343915 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:28.344309 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:28.344710 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:28.346351 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:28.347827 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:28.349672 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:28.350376 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:28.350516 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:28.355271 (+ 4755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:28.355280 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:28.355301 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:28.355356 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:28.358559 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:28.359376 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:28.359384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:28.359391 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:28.359554 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:28.359993 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:28.359999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:28.360003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:28.360886 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:28.361246 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:28.361770 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 00:21:28.362241 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":375,"threads_started":1}
I20260811 00:21:28.365003 4319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f: No bootstrap required, opened a new log
I20260811 00:21:28.373421 4319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c194119d5fd4040acaec77eefcecd4f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40705 } }
I20260811 00:21:28.373852 4319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:21:28.373963 4319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c194119d5fd4040acaec77eefcecd4f, State: Initialized, Role: FOLLOWER
I20260811 00:21:28.374629 4319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c194119d5fd4040acaec77eefcecd4f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40705 } }
I20260811 00:21:28.374866 4319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:21:28.375033 4319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:21:28.375236 4319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:21:28.377781 4319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c194119d5fd4040acaec77eefcecd4f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40705 } }
I20260811 00:21:28.378355 4319 leader_election.cc:302] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c194119d5fd4040acaec77eefcecd4f; no voters:
I20260811 00:21:28.379002 4319 leader_election.cc:288] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:21:28.379163 4327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:21:28.380012 4327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [term 1 LEADER]: Becoming Leader. State: Replica: 5c194119d5fd4040acaec77eefcecd4f, State: Running, Role: LEADER
I20260811 00:21:28.380664 4327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c194119d5fd4040acaec77eefcecd4f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40705 } }
I20260811 00:21:28.381347 4319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:21:28.383536 4329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c194119d5fd4040acaec77eefcecd4f. Latest consensus state: current_term: 1 leader_uuid: "5c194119d5fd4040acaec77eefcecd4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c194119d5fd4040acaec77eefcecd4f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40705 } } }
I20260811 00:21:28.384058 4329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [sys.catalog]: This master's current role is: LEADER
I20260811 00:21:28.383982 4328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c194119d5fd4040acaec77eefcecd4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c194119d5fd4040acaec77eefcecd4f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40705 } } }
I20260811 00:21:28.384267 4328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f [sys.catalog]: This master's current role is: LEADER
I20260811 00:21:28.385169 4346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:21:28.392366 4346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:21:28.398825 4319 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 00:21:28.410462 4346 catalog_manager.cc:1347] Generated new cluster ID: 633e41d33f82489abf7bfcee2ad1047c
I20260811 00:21:28.410936 4346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:21:28.441061 4346 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:21:28.442718 4346 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:21:28.455927 4346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f: Generated new TSK 0
I20260811 00:21:28.457103 4346 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:21:29.293522 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:40111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40705
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:21:29 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 00:21:29.793807 4407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:21:29.794392 4407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:21:29.794633 4407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:21:29.807644 4407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:21:29.807897 4407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:21:29.808140 4407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:21:29.825580 4407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.71.148:40111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:21:29.827536 4407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:21:29.829843 4407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:21:29.850603 4426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:21:29.851475 4427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:21:29.854660 4407 server_base.cc:1034] running on GCE node
W20260811 00:21:29.857447 4431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:21:29.858803 4407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:21:29.861567 4407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:21:29.865163 4407 hybrid_clock.cc:648] HybridClock initialized: now 1786407689864946 us; error 218 us; skew 500 ppm
I20260811 00:21:29.865799 4407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:21:29 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:21:29.871457 4407 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:21:29.874019 4407 webserver.cc:460] Webserver started at http://127.16.71.129:43033/ using document root <none> and password file <none>
I20260811 00:21:29.875274 4407 fs_manager.cc:359] Metadata directory not provided
I20260811 00:21:29.875512 4407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:21:29.876004 4407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:21:29 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:21:29.907459 4407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c"
format_stamp: "Formatted at 2026-08-11 00:21:29 on dist-test-slave-f7th"
server_key: "3440e5e0fe78978c8eb53ff2d4b4543b"
server_key_iv: "07add97b5baa24ab07a0c34c3887690f"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:29.909659 4407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c"
format_stamp: "Formatted at 2026-08-11 00:21:29 on dist-test-slave-f7th"
server_key: "3440e5e0fe78978c8eb53ff2d4b4543b"
server_key_iv: "07add97b5baa24ab07a0c34c3887690f"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:29.919624 4407 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.004s
I20260811 00:21:29.956789 4439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:29.958803 4407 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.007s sys 0.009s
I20260811 00:21:29.959205 4407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c"
format_stamp: "Formatted at 2026-08-11 00:21:29 on dist-test-slave-f7th"
server_key: "3440e5e0fe78978c8eb53ff2d4b4543b"
server_key_iv: "07add97b5baa24ab07a0c34c3887690f"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:29.959547 4407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:21:30.010229 4407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:21:30.016587 4407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:21:30.017021 4407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:21:30.018484 4407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:21:30.020545 4407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:21:30.020671 4407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 00:21:30.020807 4407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:21:30.020879 4407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:21:30 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:21:30.227507 4407 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:35755
I20260811 00:21:30.227514 4658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:35755 every 8 connection(s)
I20260811 00:21:30.229573 4407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:21:30.238127 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 4407
I20260811 00:21:30.238365 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:21:30.239223 16670 external_mini_cluster.cc:1428] Setting key 1e6acfcad452bda6a49f15d8fe9e7e11
Aug 11 00:21:30 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:21:30.274778 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:30.233905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40705 (local address 127.16.71.129:40685)
0811 00:21:30.249335 (+ 15430us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:30.249406 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:30.252092 (+ 2686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:30.253415 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:30.253430 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:30.254117 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:21:30.255044 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:30.255073 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:30.256780 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:30.256786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:30.258914 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:30.258942 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:30.260359 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:30.260367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:30.260732 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:30.261620 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:21:30.261654 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:21:30.264683 (+ 3029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:21:30.269179 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:21:30.269190 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:21:30.269212 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:21:30.269875 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:21:30.270386 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:21:30.270391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:21:30.270394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:21:30.270756 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:21:30.271566 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:21:30.271576 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:21:30.272116 (+ 540us) client_negotiation.cc:769] Sending connection context
0811 00:21:30.272475 (+ 359us) client_negotiation.cc:240] Negotiation successful
0811 00:21:30.273232 (+ 757us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15038,"thread_start_us":14771,"threads_started":1}
I20260811 00:21:30.276957 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:30.233056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:40685 (local address 127.16.71.190:40705)
0811 00:21:30.250067 (+ 17011us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:30.250077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:30.250112 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:21:30.252663 (+ 2551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:30.252670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:30.252764 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:30.252866 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:30.255307 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:30.256514 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:30.259149 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:30.259828 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:30.259935 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:30.265032 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:30.265041 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:30.265046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:30.265094 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:30.268954 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:30.270074 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:30.270080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:30.270084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:30.270253 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:30.270916 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:30.270921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:30.270925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:30.271358 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:30.276053 (+ 4695us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:30.276226 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 00:21:30.276493 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15399,"thread_start_us":17050,"threads_started":1}
I20260811 00:21:30.278683 4660 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40705
I20260811 00:21:30.279057 4660 heartbeater.cc:464] Registering TS with master...
I20260811 00:21:30.280033 4660 heartbeater.cc:511] Master 127.16.71.190:40705 requested a full tablet report, sending...
I20260811 00:21:30.282953 4260 ts_manager.cc:194] Registered new tserver with Master: 0dcbdb9f50f542d7b8a3701fd345ba7c (127.16.71.129:35755)
I20260811 00:21:30.285328 4260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:40685
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:21:30.338025 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:40111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40705
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:21:30 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
W20260811 00:21:30.830506 4674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:21:30.830860 4674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:21:30.830968 4674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:21:30.838671 4674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:21:30.838816 4674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:21:30.838967 4674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:21:30.850046 4674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.71.148:40111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:21:30.851755 4674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:21:30.853478 4674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:21:30.869071 4696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:21:30.869156 4697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:21:30.869109 4701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:21:30.870301 4674 server_base.cc:1034] running on GCE node
I20260811 00:21:30.871112 4674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:21:30.872215 4674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:21:30.873476 4674 hybrid_clock.cc:648] HybridClock initialized: now 1786407690873374 us; error 62 us; skew 500 ppm
I20260811 00:21:30.873862 4674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:21:30 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407690, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:21:30.877656 4674 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:21:30.879386 4674 webserver.cc:460] Webserver started at http://127.16.71.130:35671/ using document root <none> and password file <none>
I20260811 00:21:30.880187 4674 fs_manager.cc:359] Metadata directory not provided
I20260811 00:21:30.880327 4674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:21:30.880736 4674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:21:30 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407690, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:21:30.896497 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "eabe3c8cd5e742edad03a711f6d53c4d"
format_stamp: "Formatted at 2026-08-11 00:21:30 on dist-test-slave-f7th"
server_key: "2828f4df39ee19f7b505892059fb73e3"
server_key_iv: "119753bc5da167aff8036ca457fecea5"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:30.897727 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "eabe3c8cd5e742edad03a711f6d53c4d"
format_stamp: "Formatted at 2026-08-11 00:21:30 on dist-test-slave-f7th"
server_key: "2828f4df39ee19f7b505892059fb73e3"
server_key_iv: "119753bc5da167aff8036ca457fecea5"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:30.904217 4674 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:21:30.923897 4722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:30.925999 4674 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:21:30.926353 4674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "eabe3c8cd5e742edad03a711f6d53c4d"
format_stamp: "Formatted at 2026-08-11 00:21:30 on dist-test-slave-f7th"
server_key: "2828f4df39ee19f7b505892059fb73e3"
server_key_iv: "119753bc5da167aff8036ca457fecea5"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:30.926749 4674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:21:30.958788 4674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:21:30.963774 4674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:21:30.964129 4674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:21:30.965461 4674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:21:30.967374 4674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:21:30.967480 4674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:30.967608 4674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:21:30.967672 4674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:31.014801 4674 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:35713
I20260811 00:21:31.014812 4918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:35713 every 8 connection(s)
I20260811 00:21:31.016804 4674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:21:31.023267 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 4674
I20260811 00:21:31.023483 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:21:31.024063 16670 external_mini_cluster.cc:1428] Setting key 0202def513c433dd9f2fa30a73d159c9
Aug 11 00:21:31 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407690, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:21:31.042428 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.020350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:54823 (local address 127.16.71.190:40705)
0811 00:21:31.021052 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.021059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.022076 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 00:21:31.024767 (+ 2691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.024774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.024903 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.025050 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:31.027926 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.028780 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.030600 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.031038 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.031172 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.035294 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:31.035300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:31.035303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:31.035346 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:31.038284 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:31.039232 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:31.039237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:31.039240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:31.039396 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:31.039943 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:31.039948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:31.039950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:31.040373 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:31.040564 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.041763 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.042043 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":346,"threads_started":1}
I20260811 00:21:31.043843 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.020961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40705 (local address 127.16.71.130:54823)
0811 00:21:31.021861 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:31.021920 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:31.024325 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:31.025692 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:31.025707 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:31.026421 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:21:31.027662 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.027689 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.028988 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.028992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.030397 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.030415 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.031333 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.031343 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.031604 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.032352 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:21:31.032386 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:21:31.034944 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:21:31.038508 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:21:31.038519 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:21:31.038538 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:21:31.039071 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:21:31.039530 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:21:31.039536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:21:31.039540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:21:31.039799 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:21:31.040590 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:21:31.040601 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:21:31.041200 (+ 599us) client_negotiation.cc:769] Sending connection context
0811 00:21:31.041598 (+ 398us) client_negotiation.cc:240] Negotiation successful
0811 00:21:31.042273 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":376,"threads_started":1}
I20260811 00:21:31.046274 4919 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40705
I20260811 00:21:31.046681 4919 heartbeater.cc:464] Registering TS with master...
I20260811 00:21:31.047821 4919 heartbeater.cc:511] Master 127.16.71.190:40705 requested a full tablet report, sending...
I20260811 00:21:31.050004 4260 ts_manager.cc:194] Registered new tserver with Master: eabe3c8cd5e742edad03a711f6d53c4d (127.16.71.130:35713)
I20260811 00:21:31.051226 4260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:54823
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:21:31.100844 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:40111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40705
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:21:31.289986 4660 heartbeater.cc:503] Master 127.16.71.190:40705 was elected leader, sending a full tablet report...
W20260811 00:21:31.494243 4933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:21:31.494566 4933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:21:31.494684 4933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:21:31.502861 4933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:21:31.503032 4933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:21:31.503201 4933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:21:31.513818 4933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.71.148:40111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:21:31.515275 4933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:21:31.516775 4933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:21:31.531203 4943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:21:31.531167 4949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:21:31.531255 4942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:21:31.532461 4933 server_base.cc:1034] running on GCE node
I20260811 00:21:31.533262 4933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:21:31.534444 4933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:21:31.535713 4933 hybrid_clock.cc:648] HybridClock initialized: now 1786407691535630 us; error 71 us; skew 500 ppm
I20260811 00:21:31.536140 4933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:21:31 dist-test-slave-f7th krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:21:31.539891 4933 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:21:31.541916 4933 webserver.cc:460] Webserver started at http://127.16.71.131:41041/ using document root <none> and password file <none>
I20260811 00:21:31.542732 4933 fs_manager.cc:359] Metadata directory not provided
I20260811 00:21:31.542891 4933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:21:31.543326 4933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:21:31 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:21:31.564213 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5"
format_stamp: "Formatted at 2026-08-11 00:21:31 on dist-test-slave-f7th"
server_key: "fda2e02ff7d849e67cc7db03e1e2803d"
server_key_iv: "873ea0277c3b0d1ebc40772d0e3f4419"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:31.565313 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5"
format_stamp: "Formatted at 2026-08-11 00:21:31 on dist-test-slave-f7th"
server_key: "fda2e02ff7d849e67cc7db03e1e2803d"
server_key_iv: "873ea0277c3b0d1ebc40772d0e3f4419"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:31.573344 4933 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 00:21:31.590741 4962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:31.592716 4933 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:21:31.593005 4933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5"
format_stamp: "Formatted at 2026-08-11 00:21:31 on dist-test-slave-f7th"
server_key: "fda2e02ff7d849e67cc7db03e1e2803d"
server_key_iv: "873ea0277c3b0d1ebc40772d0e3f4419"
server_key_version: "kuduclusterkey@0"
I20260811 00:21:31.593286 4933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:21:31.636859 4933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:21:31.641630 4933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:21:31.641983 4933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:21:31.643074 4933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:21:31.644695 4933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:21:31.644797 4933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:31.644932 4933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:21:31.644999 4933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:21:31.684118 4933 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:39097
I20260811 00:21:31.684163 5175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:39097 every 8 connection(s)
I20260811 00:21:31.685704 4933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:21:31.687330 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 4933
I20260811 00:21:31.687500 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:21:31.688020 16670 external_mini_cluster.cc:1428] Setting key d788ca05ddf263cc56edf129cbc8aa17
Aug 11 00:21:31 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:21:31.709173 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.688889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:58655 (local address 127.16.71.190:40705)
0811 00:21:31.690100 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.690107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.690884 (+ 777us) server_negotiation.cc:407] Connection header received
0811 00:21:31.693215 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.693220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.693315 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.693425 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:31.695136 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.695910 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.697729 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.698232 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.698390 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.702192 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:31.702197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:31.702201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:31.702247 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:31.705543 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:31.706510 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:31.706515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:31.706518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:31.706679 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:31.707302 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:31.707306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:31.707308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:31.707638 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:31.707769 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.708605 (+ 836us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.708889 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":921,"threads_started":1}
I20260811 00:21:31.710319 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.689644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40705 (local address 127.16.71.131:58655)
0811 00:21:31.690684 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:31.690745 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:31.692883 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:31.693638 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:31.693647 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:31.694157 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:21:31.694915 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.694939 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.696089 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.696093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.697574 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.697589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.698525 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.698533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.698765 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.699496 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:21:31.699521 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:21:31.701890 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:21:31.705719 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:21:31.705728 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:21:31.705746 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:21:31.706341 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:21:31.706838 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:21:31.706842 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:21:31.706845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:21:31.707151 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:21:31.707774 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:21:31.707782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:21:31.708192 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 00:21:31.708469 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 00:21:31.709118 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":422,"threads_started":1}
I20260811 00:21:31.711908 5176 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40705
I20260811 00:21:31.712208 5176 heartbeater.cc:464] Registering TS with master...
I20260811 00:21:31.713088 5176 heartbeater.cc:511] Master 127.16.71.190:40705 requested a full tablet report, sending...
I20260811 00:21:31.714650 4260 ts_manager.cc:194] Registered new tserver with Master: 0d0aa1b8cc8941c29b69e3fb1cca58b5 (127.16.71.131:39097)
I20260811 00:21:31.715848 4260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:58655
I20260811 00:21:31.717840 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:21:31.737376 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.723277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.16.71.190:40705)
0811 00:21:31.723523 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.723529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.723919 (+ 390us) server_negotiation.cc:407] Connection header received
0811 00:21:31.724120 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.724124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.724207 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.724296 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:31.725882 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.726645 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.728145 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.728588 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.728768 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.731352 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:31.731357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:31.731361 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:31.731401 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:31.734298 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:31.735164 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:31.735169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:31.735171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:31.735317 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:31.735727 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:31.735731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:31.735733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:31.736060 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:31.736305 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.736774 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.737049 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:21:31.742326 4260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:21:31.749900 4260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:21:31.779910 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.765199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.16.71.129:35755)
0811 00:21:31.765919 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.765926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.770544 (+ 4618us) server_negotiation.cc:407] Connection header received
0811 00:21:31.770843 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.770851 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.771054 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.771221 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:21:31.772172 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.773809 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.777196 (+ 3387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.778580 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.778722 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.778985 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.779353 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.779551 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":390,"threads_started":1}
I20260811 00:21:31.779943 5206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.765076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57760)
0811 00:21:31.770428 (+ 5352us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:31.770445 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:31.770648 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:31.771523 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:31.771527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:31.771548 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:21:31.771991 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.772004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.774001 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.774007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.777001 (+ 2994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.777016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.778811 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.778818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.779046 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.779109 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 00:21:31.779271 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 00:21:31.779477 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5218,"mutex_wait_us":644,"thread_start_us":4312,"threads_started":1}
I20260811 00:21:31.780035 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.765223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35550 (local address 127.16.71.131:39097)
0811 00:21:31.769234 (+ 4011us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.769241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.769961 (+ 720us) server_negotiation.cc:407] Connection header received
0811 00:21:31.770395 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.770406 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.770676 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.770874 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:21:31.772048 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.773807 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.777048 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.778580 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.778722 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.778985 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.779533 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.779705 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3855,"thread_start_us":3700,"threads_started":1}
I20260811 00:21:31.780881 5199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.764739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:39097 (local address 127.0.0.1:35550)
0811 00:21:31.769767 (+ 5028us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:31.769809 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:31.770071 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:31.771252 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:31.771259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:31.771288 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:21:31.771777 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.771789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.774001 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.774007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.776824 (+ 2817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.776842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.779148 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.779153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.779295 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.779332 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:21:31.780296 (+ 964us) client_negotiation.cc:240] Negotiation successful
0811 00:21:31.780467 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4800,"mutex_wait_us":4,"thread_start_us":5363,"threads_started":1}
I20260811 00:21:31.783339 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.770456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:35713 (local address 127.0.0.1:49916)
0811 00:21:31.774926 (+ 4470us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:31.774946 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:31.775090 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:31.775676 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:31.775680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:31.775705 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:21:31.776235 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.776252 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.777986 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.777991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.780467 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.780481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.782533 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.782542 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.782750 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.782779 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 00:21:31.782866 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:21:31.783011 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4331,"thread_start_us":4165,"threads_started":1}
I20260811 00:21:31.783970 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.770684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.16.71.130:35713)
0811 00:21:31.774612 (+ 3928us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.774623 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.775065 (+ 442us) server_negotiation.cc:407] Connection header received
0811 00:21:31.775255 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.775261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.775475 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.775660 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:21:31.776648 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.777823 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.780708 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.782155 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.782290 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.782508 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.782944 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.783541 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3692,"thread_start_us":3495,"threads_started":1}
I20260811 00:21:31.784672 4529 tablet_service.cc:1467] Processing CreateTablet for tablet 7b76bd74f4304ccbb8223fef8cd1477c (DEFAULT_TABLE table=test-table [id=622e8d566ee849f2909bee0684b71667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:21:31.784673 5057 tablet_service.cc:1467] Processing CreateTablet for tablet 7b76bd74f4304ccbb8223fef8cd1477c (DEFAULT_TABLE table=test-table [id=622e8d566ee849f2909bee0684b71667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:21:31.786394 5057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b76bd74f4304ccbb8223fef8cd1477c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:21:31.786383 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b76bd74f4304ccbb8223fef8cd1477c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:21:31.787367 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 7b76bd74f4304ccbb8223fef8cd1477c (DEFAULT_TABLE table=test-table [id=622e8d566ee849f2909bee0684b71667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:21:31.788709 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b76bd74f4304ccbb8223fef8cd1477c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:21:31.801563 5210 tablet_bootstrap.cc:492] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Bootstrap starting.
I20260811 00:21:31.805539 5212 tablet_bootstrap.cc:492] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Bootstrap starting.
I20260811 00:21:31.806686 5210 tablet_bootstrap.cc:654] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Neither blocks nor log segments found. Creating new log.
I20260811 00:21:31.807574 5213 tablet_bootstrap.cc:492] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Bootstrap starting.
I20260811 00:21:31.808889 5210 log.cc:824] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:21:31.809792 5212 tablet_bootstrap.cc:654] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Neither blocks nor log segments found. Creating new log.
I20260811 00:21:31.811256 5212 log.cc:824] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:21:31.811455 5213 tablet_bootstrap.cc:654] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Neither blocks nor log segments found. Creating new log.
I20260811 00:21:31.812774 5213 log.cc:824] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:21:31.813513 5210 tablet_bootstrap.cc:492] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: No bootstrap required, opened a new log
I20260811 00:21:31.814395 5210 ts_tablet_manager.cc:1397] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260811 00:21:31.815295 5212 tablet_bootstrap.cc:492] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: No bootstrap required, opened a new log
I20260811 00:21:31.815616 5212 ts_tablet_manager.cc:1397] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 00:21:31.819218 5213 tablet_bootstrap.cc:492] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: No bootstrap required, opened a new log
I20260811 00:21:31.819530 5213 ts_tablet_manager.cc:1397] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 00:21:31.826738 5212 raft_consensus.cc:348] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.826738 5210 raft_consensus.cc:348] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.827322 5212 raft_consensus.cc:374] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:21:31.827322 5210 raft_consensus.cc:374] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:21:31.827447 5212 raft_consensus.cc:729] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcbdb9f50f542d7b8a3701fd345ba7c, State: Initialized, Role: FOLLOWER
I20260811 00:21:31.827447 5210 raft_consensus.cc:729] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eabe3c8cd5e742edad03a711f6d53c4d, State: Initialized, Role: FOLLOWER
I20260811 00:21:31.827958 5213 raft_consensus.cc:348] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.828320 5210 consensus_queue.cc:260] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.828325 5212 consensus_queue.cc:260] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.828771 5213 raft_consensus.cc:374] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:21:31.828981 5213 raft_consensus.cc:729] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d0aa1b8cc8941c29b69e3fb1cca58b5, State: Initialized, Role: FOLLOWER
I20260811 00:21:31.830089 5213 consensus_queue.cc:260] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.831768 5212 ts_tablet_manager.cc:1428] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260811 00:21:31.833421 4919 heartbeater.cc:503] Master 127.16.71.190:40705 was elected leader, sending a full tablet report...
I20260811 00:21:31.834596 5176 heartbeater.cc:503] Master 127.16.71.190:40705 was elected leader, sending a full tablet report...
I20260811 00:21:31.835243 5213 ts_tablet_manager.cc:1428] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Time spent starting tablet: real 0.016s user 0.005s sys 0.011s
I20260811 00:21:31.836102 5210 ts_tablet_manager.cc:1428] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Time spent starting tablet: real 0.021s user 0.014s sys 0.005s
I20260811 00:21:31.842414 5221 raft_consensus.cc:482] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:21:31.842819 5221 raft_consensus.cc:504] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.845148 5221 leader_election.cc:288] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d0aa1b8cc8941c29b69e3fb1cca58b5 (127.16.71.131:39097), 0dcbdb9f50f542d7b8a3701fd345ba7c (127.16.71.129:35755)
I20260811 00:21:31.857191 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.845776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:52469 (local address 127.16.71.129:35755)
0811 00:21:31.846047 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.846051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.847571 (+ 1520us) server_negotiation.cc:407] Connection header received
0811 00:21:31.847836 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.847842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.847928 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.848056 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:21:31.849267 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.850807 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.854374 (+ 3567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.855927 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.856083 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.856165 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.856752 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.856915 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260811 00:21:31.857206 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.845662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:38517 (local address 127.16.71.131:39097)
0811 00:21:31.845914 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.845921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.847602 (+ 1681us) server_negotiation.cc:407] Connection header received
0811 00:21:31.847836 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.847842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.847928 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.848049 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:21:31.849267 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.850667 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.854324 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.855848 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.856080 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.856165 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.856748 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.856928 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 00:21:31.857671 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.845687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.16.71.130:52469)
0811 00:21:31.847386 (+ 1699us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:31.847410 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:31.847675 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:31.848127 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:31.848133 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:31.848190 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:21:31.848976 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.848997 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.850965 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.850972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.853983 (+ 3011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.854059 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.856201 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.856209 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.856476 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.856549 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 00:21:31.856694 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 00:21:31.856900 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1507,"mutex_wait_us":110,"spinlock_wait_cycles":96128,"thread_start_us":1006,"threads_started":1}
I20260811 00:21:31.857976 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.845687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:39097 (local address 127.16.71.130:38517)
0811 00:21:31.847433 (+ 1746us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:31.847449 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:21:31.847675 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:21:31.848127 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:21:31.848133 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:21:31.848189 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:21:31.849045 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.849054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.850965 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.850972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.853990 (+ 3018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:21:31.854016 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.856201 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.856209 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:21:31.856483 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.856548 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 00:21:31.856694 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 00:21:31.856900 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1636,"mutex_wait_us":249,"spinlock_wait_cycles":19712,"thread_start_us":1212,"threads_started":1}
I20260811 00:21:31.859014 5091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b76bd74f4304ccbb8223fef8cd1477c" candidate_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" is_pre_election: true
I20260811 00:21:31.859026 4574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b76bd74f4304ccbb8223fef8cd1477c" candidate_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" is_pre_election: true
I20260811 00:21:31.859685 5091 raft_consensus.cc:2454] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eabe3c8cd5e742edad03a711f6d53c4d in term 0.
I20260811 00:21:31.859685 4574 raft_consensus.cc:2454] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eabe3c8cd5e742edad03a711f6d53c4d in term 0.
I20260811 00:21:31.860752 4729 leader_election.cc:302] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcbdb9f50f542d7b8a3701fd345ba7c, eabe3c8cd5e742edad03a711f6d53c4d; no voters:
I20260811 00:21:31.861294 5221 raft_consensus.cc:2781] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:21:31.861509 5221 raft_consensus.cc:482] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:21:31.861645 5221 raft_consensus.cc:3037] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:21:31.865620 5221 raft_consensus.cc:504] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.866489 5221 leader_election.cc:288] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [CANDIDATE]: Term 1 election: Requested vote from peers 0d0aa1b8cc8941c29b69e3fb1cca58b5 (127.16.71.131:39097), 0dcbdb9f50f542d7b8a3701fd345ba7c (127.16.71.129:35755)
I20260811 00:21:31.867264 4574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b76bd74f4304ccbb8223fef8cd1477c" candidate_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c"
I20260811 00:21:31.867323 5091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b76bd74f4304ccbb8223fef8cd1477c" candidate_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5"
I20260811 00:21:31.867540 4574 raft_consensus.cc:3037] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:21:31.867606 5091 raft_consensus.cc:3037] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:21:31.870992 4574 raft_consensus.cc:2454] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eabe3c8cd5e742edad03a711f6d53c4d in term 1.
I20260811 00:21:31.871101 5091 raft_consensus.cc:2454] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eabe3c8cd5e742edad03a711f6d53c4d in term 1.
I20260811 00:21:31.871822 4732 leader_election.cc:302] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d0aa1b8cc8941c29b69e3fb1cca58b5, 0dcbdb9f50f542d7b8a3701fd345ba7c, eabe3c8cd5e742edad03a711f6d53c4d; no voters:
I20260811 00:21:31.872663 5221 raft_consensus.cc:2781] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:21:31.874224 5221 raft_consensus.cc:686] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 1 LEADER]: Becoming Leader. State: Replica: eabe3c8cd5e742edad03a711f6d53c4d, State: Running, Role: LEADER
I20260811 00:21:31.875139 5221 consensus_queue.cc:237] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } }
I20260811 00:21:31.882551 4257 catalog_manager.cc:5495] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to eabe3c8cd5e742edad03a711f6d53c4d (127.16.71.130). New cstate: current_term: 1 leader_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eabe3c8cd5e742edad03a711f6d53c4d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35713 } health_report { overall_health: HEALTHY } } }
I20260811 00:21:31.917099 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.910122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.16.71.130:35713)
0811 00:21:31.910376 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.910382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.910403 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:21:31.910519 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.910522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.910605 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.910756 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:21:31.911432 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.912283 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.914090 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.914751 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.914891 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:31.915534 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:21:31.916156 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:21:31.916321 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:31.916513 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 00:21:31.916653 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:21:31.929039 4574 raft_consensus.cc:1261] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 1 FOLLOWER]: Refusing update from remote peer eabe3c8cd5e742edad03a711f6d53c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:21:31.929014 5091 raft_consensus.cc:1261] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 1 FOLLOWER]: Refusing update from remote peer eabe3c8cd5e742edad03a711f6d53c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:21:31.930761 5226 consensus_queue.cc:1035] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dcbdb9f50f542d7b8a3701fd345ba7c" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 00:21:31.931560 5221 consensus_queue.cc:1035] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d0aa1b8cc8941c29b69e3fb1cca58b5" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 39097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:21:31.937721 5177 tablet.cc:2369] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:21:31.966414 4257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:21:31.966846 4257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:21:31.971570 4257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c194119d5fd4040acaec77eefcecd4f: Sending DeleteTablet for 3 replicas of tablet 7b76bd74f4304ccbb8223fef8cd1477c
I20260811 00:21:31.973228 5057 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b76bd74f4304ccbb8223fef8cd1477c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:21:31 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:35550
I20260811 00:21:31.973381 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b76bd74f4304ccbb8223fef8cd1477c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:21:31 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:49916
I20260811 00:21:31.974769 4529 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b76bd74f4304ccbb8223fef8cd1477c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:21:31 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:57760
I20260811 00:21:31.975780 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 4407
I20260811 00:21:31.992396 5256 tablet_replica.cc:330] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: stopping tablet replica
I20260811 00:21:31.993181 5255 tablet_replica.cc:330] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: stopping tablet replica
I20260811 00:21:31.993646 5255 raft_consensus.cc:2229] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:21:31.994325 5255 raft_consensus.cc:2258] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:21:31.997198 5256 raft_consensus.cc:2229] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:21:31.997393 5254 tablet_replica.cc:330] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: stopping tablet replica
I20260811 00:21:31.997825 5254 raft_consensus.cc:2229] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:21:31.997874 5256 raft_consensus.cc:2258] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:21:31.998381 5254 raft_consensus.cc:2258] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:21:32.000180 5255 ts_tablet_manager.cc:1905] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:21:32.001148 5256 ts_tablet_manager.cc:1905] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:21:32.002636 5254 ts_tablet_manager.cc:1905] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:21:31 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:21:32.008842 5255 ts_tablet_manager.cc:1918] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:21:32.009092 5255 log.cc:1196] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/7b76bd74f4304ccbb8223fef8cd1477c
I20260811 00:21:32.009675 5255 ts_tablet_manager.cc:1939] T 7b76bd74f4304ccbb8223fef8cd1477c P eabe3c8cd5e742edad03a711f6d53c4d: Deleting consensus metadata
I20260811 00:21:32.011858 4228 catalog_manager.cc:4841] TS eabe3c8cd5e742edad03a711f6d53c4d (127.16.71.130:35713): tablet 7b76bd74f4304ccbb8223fef8cd1477c (table test-table [id=622e8d566ee849f2909bee0684b71667]) successfully deleted
I20260811 00:21:32.014307 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:31.977660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.16.71.129:35755)
0811 00:21:31.977879 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:31.977886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:31.988796 (+ 10910us) server_negotiation.cc:407] Connection header received
0811 00:21:31.989446 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:31.989454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:31.989564 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:31.989649 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:31.994854 (+ 5205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.995671 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.997712 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:31.998597 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:31.998753 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:32.004660 (+ 5907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:32.004680 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:32.004700 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:32.004767 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:32.009768 (+ 5001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:32.011076 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:32.011088 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:32.011092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:32.011296 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:32.011913 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:32.011918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:32.011922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:32.012480 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:32.012871 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:32.013591 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 00:21:32.013876 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:21:32.015290 5256 ts_tablet_manager.cc:1918] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:21:32.015863 4650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:21:32.021593 5256 log.cc:1196] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/7b76bd74f4304ccbb8223fef8cd1477c
I20260811 00:21:32.024559 5254 ts_tablet_manager.cc:1918] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:21:32.024995 5254 log.cc:1196] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/7b76bd74f4304ccbb8223fef8cd1477c
I20260811 00:21:32.025864 5254 ts_tablet_manager.cc:1939] T 7b76bd74f4304ccbb8223fef8cd1477c P 0d0aa1b8cc8941c29b69e3fb1cca58b5: Deleting consensus metadata
I20260811 00:21:32.028340 4229 catalog_manager.cc:4841] TS 0d0aa1b8cc8941c29b69e3fb1cca58b5 (127.16.71.131:39097): tablet 7b76bd74f4304ccbb8223fef8cd1477c (table test-table [id=622e8d566ee849f2909bee0684b71667]) successfully deleted
I20260811 00:21:32.143919 5256 ts_tablet_manager.cc:1939] T 7b76bd74f4304ccbb8223fef8cd1477c P 0dcbdb9f50f542d7b8a3701fd345ba7c: Deleting consensus metadata
I20260811 00:21:32.144187 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 4407
I20260811 00:21:32.172940 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 4674
W20260811 00:21:32.173025 4228 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.71.129:35755 (error 108)
I20260811 00:21:32.175150 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.174631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57778)
0811 00:21:32.174827 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.174956 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:21:32.175462 4228 catalog_manager.cc:4584] TS 0dcbdb9f50f542d7b8a3701fd345ba7c (127.16.71.129:35755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b76bd74f4304ccbb8223fef8cd1477c: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Aug 11 00:21:32 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:21:32.193230 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.174178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.16.71.130:35713)
0811 00:21:32.174369 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:32.174375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:32.174434 (+ 59us) server_negotiation.cc:407] Connection header received
0811 00:21:32.174506 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:32.174512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:32.174644 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:32.174747 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:32.176359 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:32.177900 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:32.179336 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:32.180045 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:32.180187 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:32.184881 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:32.184897 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:32.184925 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:32.185001 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:32.189441 (+ 4440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:32.190458 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:32.190473 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:32.190477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:32.190727 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:32.191153 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:32.191158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:32.191161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:32.191842 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:32.192224 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:32.192487 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 00:21:32.192817 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:21:32.194130 4914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:21:32.229458 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.228827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57788)
0811 00:21:32.229104 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.229241 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 00:21:32.230971 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.230513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57804)
0811 00:21:32.230684 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.230786 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 00:21:32.291890 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.291321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57806)
0811 00:21:32.291581 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.291690 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 00:21:32.293067 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.292667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57808)
0811 00:21:32.292851 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.292933 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 00:21:32.314967 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 4674
I20260811 00:21:32.341524 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 4933
Aug 11 00:21:32 dist-test-slave-f7th krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:21:32.358431 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.342365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35566 (local address 127.16.71.131:39097)
0811 00:21:32.342551 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:21:32.342557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:21:32.342578 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:21:32.342808 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:21:32.342813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:21:32.342891 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:21:32.342985 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:21:32.344391 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:32.345225 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:32.346582 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:21:32.346995 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:21:32.347119 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:21:32.352173 (+ 5054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:21:32.352192 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:21:32.352211 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:21:32.352254 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:21:32.355244 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:32.356125 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:32.356133 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:32.356136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:32.356290 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:21:32.356806 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:21:32.356810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:21:32.356813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:21:32.357263 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:21:32.357520 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 00:21:32.357912 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 00:21:32.358122 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:21:32.359154 5167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:21:32.373705 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.373183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57820)
0811 00:21:32.373402 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.373504 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 00:21:32.374853 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.374440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57824)
0811 00:21:32.374648 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.374726 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 00:21:32.469027 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 4933
I20260811 00:21:32.495405 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 4195
I20260811 00:21:32.496961 4313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:21:32.612551 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:21:32.611971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35755 (local address 127.0.0.1:57834)
0811 00:21:32.612229 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 00:21:32.612341 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 00:21:32.612608 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 4195
I20260811 00:21:32.645705 16670 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 00:21:32.993198 16670 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 00:21:32.993535 16670 mini_ranger.cc:66] Stopping Ranger...
I20260811 00:21:33.350790 16670 mini_ranger.cc:68] Stopped Ranger
2026-08-11 00:21:33.351 UTC [3422] LOG: received smart shutdown request
2026-08-11 00:21:33.355 UTC [3422] LOG: background worker "logical replication launcher" (PID 3441) exited with exit code 1
2026-08-11 00:21:33.356 UTC [3430] LOG: shutting down
2026-08-11 00:21:33.542 UTC [3422] LOG: database system is shut down
2026-08-11T00:21:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56738 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:21:33 dist-test-slave-f7th krb5kdc[5283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:21:33 dist-test-slave-f7th krb5kdc[5283](info): set up 2 sockets
Aug 11 00:21:33 dist-test-slave-f7th krb5kdc[5283](info): commencing operation
krb5kdc: starting...
W20260811 00:21:34.974351 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:21:35 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:21:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:21:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/rangeradmin_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/rangeradmin_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/rangerlookup_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/rangerlookup_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
I20260811 00:21:35.408852 16670 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9TQxI2/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/postgres -l logfile start
2026-08-11 00:21:38.308 UTC [5345] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 00:21:38.308 UTC [5345] LOG: listening on IPv4 address "127.16.71.148", port 51141
2026-08-11 00:21:38.312 UTC [5345] LOG: listening on Unix socket "/tmp/.s.PGSQL.51141"
2026-08-11 00:21:38.337 UTC [5348] LOG: database system was shut down at 2026-08-11 00:21:36 UTC
2026-08-11 00:21:38.343 UTC [5345] LOG: database system is ready to accept connections
I20260811 00:21:39.494236 16670 mini_postgres.cc:96] Postgres bound to 51141
2026-08-11 00:21:39.518 UTC [5369] FATAL: database "slave" does not exist
127.16.71.148:51141 - accepting connections
I20260811 00:21:39.519336 16670 mini_ranger.cc:161] Starting Ranger...
I20260811 00:21:39.552780 16670 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 00:21:40.164887 16670 mini_ranger.cc:87] Created ranger Postgres database
I20260811 00:21:40.165006 16670 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-admin
2026-08-11 00:21:40,474 [I] DB FLAVOR :POSTGRES
2026-08-11 00:21:40,475 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:21:40,475 [I] Checking connection..
2026-08-11 00:21:40,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 00:21:40,747 [I] Checking connection passed.
2026-08-11 00:21:40,747 [I] --------- Verifying version history table ---------
2026-08-11 00:21:40,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:21:41,024 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:21:41,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:21:41,300 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:21:41,300 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 00:21:41,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 00:21:41.577 UTC [5476] WARNING: there is no transaction in progress
2026-08-11 00:21:41,588 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 00:21:41,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:21:41,874 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 00:21:41,874 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:21:41,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 00:21:42,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 00:21:42,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-11 00:21:42,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:21:42,999 [I] Table x_portal_user does not exist in database ranger
2026-08-11 00:21:43,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:21:43,292 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 00:21:43,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:21:43,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:21:43,833 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 00:21:43,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 00:21:44.968 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:44.987 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.007 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.027 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.045 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.066 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.134 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.143 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.160 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.172 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.184 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.201 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.213 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.221 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.230 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:45.239 UTC [5672] WARNING: there is no transaction in progress
2026-08-11 00:21:46,110 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 00:21:46,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 00:21:46,397 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 00:21:46,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:21:46,684 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 00:21:46,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:21:46,984 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 00:21:46,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:21:47,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:21:47,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-11 00:21:47,820 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 00:21:47,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 00:21:48,099 [I] DB_PATCHES have already been applied
I20260811 00:21:48.105973 16670 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-admin:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9TQxI2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 00:21:48.106112 16670 mini_ranger.cc:193] Using host: 127.16.71.148
I20260811 00:21:48.127030 16670 mini_ranger.cc:227] Ranger admin URL: http://127.16.71.148:48333
Aug 11, 2026 12:21:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:21:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:21:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:21:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:21:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:21:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:21:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 00:21:49.517763 16670 mini_ranger.cc:160] Time spent starting Ranger: real 9.998s user 0.007s sys 0.058s
Aug 11, 2026 12:22:10 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:22:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:22:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:22:11 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:22:12 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 00:22:12.653806 16670 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/rangerkms_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/rangerkms_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.71.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.71.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.71.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/HTTP_127.16.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 00:22:12 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407732, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 00:22:12.890599 16670 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 00:22:12.927968 16670 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 00:22:13.925849 16670 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 00:22:13.926047 16670 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms
I20260811 00:22:13.926210 16670 mini_ranger_kms.cc:227] Using postgres at 127.16.71.148:51141
2026-08-11 00:22:14,301 [I] DB FLAVOR :POSTGRES
2026-08-11 00:22:14,301 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:22:14,301 [I] Checking connection
2026-08-11 00:22:14,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 00:22:14,593 [I] connection success
2026-08-11 00:22:14,593 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 00:22:14,593 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 00:22:14,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 00:22:14,882 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 00:22:14,883 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:22:14,883 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 00:22:14,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.71.148:51141/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 00:22:15,179 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 00:22:15.731473 16670 mini_ranger_kms.cc:310] Created kms service
I20260811 00:22:15.973101 16670 mini_ranger_kms.cc:326] Created kudu user
I20260811 00:22:16.026448 16670 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 00:22:16 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407732, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:16.494820 16670 mini_ranger_kms.cc:384] Added ranger policy
I20260811 00:22:16.495209 16670 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms:/tmp/dist-test-task9TQxI2/build/asan/bin/postgresql.jar:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9TQxI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 00:22:16.495280 16670 mini_ranger_kms.cc:240] Using host: 127.16.71.148
I20260811 00:22:16.519213 16670 mini_ranger_kms.cc:276] Ranger KMS PID: 6022
I20260811 00:22:16.519459 16670 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.71.148:49053
Aug 11, 2026 12:22:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:22:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:22:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:22:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:22:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:22:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 00:22:18 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.71.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 00:22:19 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.71.148@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:19.305878 16670 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.415s user 0.057s sys 0.366s
I20260811 00:22:19.306138 16670 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 00:22:19.306306 16670 mini_ranger_kms.cc:401] 127.16.71.148:49053/kms/v1/keys
Aug 11 00:22:19 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.71.148@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
Aug 11 00:22:22 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:22:22.374264 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34273
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:34273
--ranger_config_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:22.782452 6105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:22.782925 6105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:22.783118 6105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:22.793592 6105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:22.793828 6105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:22.793915 6105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:22.793996 6105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:22.794062 6105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:22.811146 6105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.71.148:49053/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:34273
--ranger_config_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:22.813246 6105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:22.814951 6105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:22.831128 6116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:22.831245 6115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:22.831266 6118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:22.832055 6105 server_base.cc:1034] running on GCE node
I20260811 00:22:22.833318 6105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:22.835357 6105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:22.836701 6105 hybrid_clock.cc:648] HybridClock initialized: now 1786407742836579 us; error 59 us; skew 500 ppm
I20260811 00:22:22.837186 6105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:22 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407742, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:22.841075 6105 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:22:22.843137 6105 webserver.cc:460] Webserver started at http://127.16.71.190:42767/ using document root <none> and password file <none>
I20260811 00:22:22.843973 6105 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:22.844120 6105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:22.844482 6105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:22:22 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407742, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:22.916213 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "2d358568f8cd47b1832ad035aea71b0b"
format_stamp: "Formatted at 2026-08-11 00:22:22 on dist-test-slave-f7th"
server_key: "843ba82369b3924c641f734a709fb257"
server_key_iv: "fe5aaff08b32db8bec13b202de72450d"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:22.917371 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "2d358568f8cd47b1832ad035aea71b0b"
format_stamp: "Formatted at 2026-08-11 00:22:22 on dist-test-slave-f7th"
server_key: "843ba82369b3924c641f734a709fb257"
server_key_iv: "fe5aaff08b32db8bec13b202de72450d"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:22.924196 6105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:22:22.948370 6136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:22.950121 6105 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:22:22.950349 6105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "2d358568f8cd47b1832ad035aea71b0b"
format_stamp: "Formatted at 2026-08-11 00:22:22 on dist-test-slave-f7th"
server_key: "843ba82369b3924c641f734a709fb257"
server_key_iv: "fe5aaff08b32db8bec13b202de72450d"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:22.950598 6105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:22.970993 6105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:22.975580 6105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:22.975919 6105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:22.996241 6105 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:34273
I20260811 00:22:22.996241 6229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:34273 every 8 connection(s)
I20260811 00:22:22.998100 6105 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:22:23.003108 6232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:23.003841 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 6105
I20260811 00:22:23.004099 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:22:23.005097 16670 external_mini_cluster.cc:1428] Setting key ae1182094399b8664e3559605ab5987d
I20260811 00:22:23.020736 6232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Bootstrap starting.
I20260811 00:22:23.025648 6232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:23.027204 6232 log.cc:824] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:23.031370 6232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: No bootstrap required, opened a new log
Aug 11 00:22:23 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:23.040903 6232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:23.041417 6232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:23.041540 6232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d358568f8cd47b1832ad035aea71b0b, State: Initialized, Role: FOLLOWER
I20260811 00:22:23.042444 6232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:23.042732 6232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:22:23.042886 6232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:22:23.043110 6232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:23.045814 6232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:23.046567 6232 leader_election.cc:302] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d358568f8cd47b1832ad035aea71b0b; no voters:
I20260811 00:22:23.047372 6232 leader_election.cc:288] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:22:23.048954 6242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:22:23.050256 6242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 LEADER]: Becoming Leader. State: Replica: 2d358568f8cd47b1832ad035aea71b0b, State: Running, Role: LEADER
I20260811 00:22:23.050922 6242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:23.051394 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:23.007769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47314 (local address 127.16.71.190:34273)
0811 00:22:23.023417 (+ 15648us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:23.023436 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:23.023496 (+ 60us) server_negotiation.cc:407] Connection header received
0811 00:22:23.026246 (+ 2750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:23.026284 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:23.026922 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:23.027457 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:23.029515 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:23.031630 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:23.033735 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:23.034960 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:23.035196 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:23.040761 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:23.040776 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:23.040803 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:23.040884 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:23.045365 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:23.046592 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:23.046604 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:23.046615 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:23.046895 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:23.047399 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:23.047404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:23.047407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:23.048317 (+ 910us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:23.048739 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:23.049348 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 00:22:23.049850 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15351,"thread_start_us":14297,"threads_started":1}
I20260811 00:22:23.053889 6254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d358568f8cd47b1832ad035aea71b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } } }
I20260811 00:22:23.054342 6254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:23.059124 6242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d358568f8cd47b1832ad035aea71b0b. Latest consensus state: current_term: 1 leader_uuid: "2d358568f8cd47b1832ad035aea71b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } } }
I20260811 00:22:23.059451 6242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:23.068171 6232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:22:23.068413 6256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:22:23.075649 6256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:22:23.089195 6256 catalog_manager.cc:1347] Generated new cluster ID: a765bbe76a0348b59e1903e2066a4c34
I20260811 00:22:23.089339 6256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:22:23.112404 6256 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:22:23.114578 6256 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:22:23.129736 6256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Generated new TSK 0
I20260811 00:22:23.132167 6256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:22:23.138482 6232 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:22:24.113708 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34273
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:22:24 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407744, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 00:22:24.544934 6320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:24.545267 6320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:24.545405 6320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:24.553889 6320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:24.554050 6320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:24.554256 6320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:22:24.566572 6320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.71.148:49053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:24.568282 6320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:24.570081 6320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:24.586781 6337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:24.587225 6339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:24.588693 6320 server_base.cc:1034] running on GCE node
W20260811 00:22:24.589511 6336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:24.590569 6320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:24.592146 6320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:24.593494 6320 hybrid_clock.cc:648] HybridClock initialized: now 1786407744593361 us; error 121 us; skew 500 ppm
I20260811 00:22:24.593961 6320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:24 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407744, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:24.598778 6320 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:22:24.607108 6320 webserver.cc:460] Webserver started at http://127.16.71.129:37893/ using document root <none> and password file <none>
I20260811 00:22:24.608260 6320 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:24.608448 6320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:24.608944 6320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:22:24 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407744, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:24.640144 6320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "ad629f3b7117495580576d9b13c84789"
format_stamp: "Formatted at 2026-08-11 00:22:24 on dist-test-slave-f7th"
server_key: "3bab14e672461704c2a19ab75af6c80e"
server_key_iv: "d44509802758bad577f193b9ee7b7b2e"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:24.644261 6320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "ad629f3b7117495580576d9b13c84789"
format_stamp: "Formatted at 2026-08-11 00:22:24 on dist-test-slave-f7th"
server_key: "3bab14e672461704c2a19ab75af6c80e"
server_key_iv: "d44509802758bad577f193b9ee7b7b2e"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:24.666921 6320 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.019s sys 0.000s
I20260811 00:22:24.705451 6355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:24.709074 6320 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.002s
I20260811 00:22:24.709340 6320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "ad629f3b7117495580576d9b13c84789"
format_stamp: "Formatted at 2026-08-11 00:22:24 on dist-test-slave-f7th"
server_key: "3bab14e672461704c2a19ab75af6c80e"
server_key_iv: "d44509802758bad577f193b9ee7b7b2e"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:24.709664 6320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:24.743505 6320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:24.748107 6320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:24.748472 6320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:24.756609 6320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:24.764223 6320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:24.764364 6320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:24.764518 6320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:24.764599 6320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:24.839778 6320 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:37467
I20260811 00:22:24.839783 6570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:37467 every 8 connection(s)
I20260811 00:22:24.841360 6320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:22:24.845675 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 6320
I20260811 00:22:24.846113 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:22:24.847149 16670 external_mini_cluster.cc:1428] Setting key 11813ecc586c3d2ee88bb09d70dce224
Aug 11 00:22:24 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407744, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
I20260811 00:22:24.894171 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:24.847017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37373 (local address 127.16.71.190:34273)
0811 00:22:24.864696 (+ 17679us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:24.864704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:24.866252 (+ 1548us) server_negotiation.cc:407] Connection header received
0811 00:22:24.868767 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:24.868773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:24.868951 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:24.869107 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:24.871588 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:24.872869 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:24.875657 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:24.876126 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:24.876255 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:24.886141 (+ 9886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:24.886149 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:24.886153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:24.886210 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:24.889396 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:24.890632 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:24.890640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:24.890643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:24.890821 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:24.891503 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:24.891508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:24.891510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:24.891849 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:24.892002 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:24.893436 (+ 1434us) server_negotiation.cc:299] Negotiation successful
0811 00:22:24.893743 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17471,"thread_start_us":17300,"threads_started":1}
I20260811 00:22:24.895632 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:24.845824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34273 (local address 127.16.71.129:37373)
0811 00:22:24.865436 (+ 19612us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:24.865532 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:24.868252 (+ 2720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:24.869483 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:24.869510 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:24.870345 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:24.871304 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:24.871336 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:24.873160 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:24.873166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:24.875428 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:24.875453 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:24.876410 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:24.876419 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:24.876812 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:24.877800 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:24.877860 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:24.885787 (+ 7927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:24.889643 (+ 3856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:24.889656 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:24.889681 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:24.890459 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:24.890980 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:24.890986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:24.890990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:24.891337 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:24.892019 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:24.892040 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:24.892680 (+ 640us) client_negotiation.cc:769] Sending connection context
0811 00:22:24.893326 (+ 646us) client_negotiation.cc:240] Negotiation successful
0811 00:22:24.893963 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19212,"thread_start_us":18662,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
I20260811 00:22:24.899298 6571 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34273
I20260811 00:22:24.899737 6571 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:24.900756 6571 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
I20260811 00:22:24.904043 6170 ts_manager.cc:194] Registered new tserver with Master: ad629f3b7117495580576d9b13c84789 (127.16.71.129:37467)
I20260811 00:22:24.906531 6170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:37373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
W20260811 00:22:24.958519 16670 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.16.71.130: real 0.108s user 0.003s sys 0.025s
I20260811 00:22:24.959762 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34273
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:22:24 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407744, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
Aug 11 00:22:25 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407744, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
W20260811 00:22:25.530431 6581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:25.530906 6581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:25.531068 6581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:25.543354 6581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:25.543538 6581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:25.543731 6581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:22:25.560142 6581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.71.148:49053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:25.562137 6581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:25.564163 6581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:25.609066 6597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:25.612727 6602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:25.613031 6593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:25.614063 6581 server_base.cc:1034] running on GCE node
I20260811 00:22:25.615111 6581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:25.625602 6581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:25.629060 6581 hybrid_clock.cc:648] HybridClock initialized: now 1786407745628894 us; error 127 us; skew 500 ppm
I20260811 00:22:25.629626 6581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:25 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407745, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:25.636037 6581 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:22:25.668283 6581 webserver.cc:460] Webserver started at http://127.16.71.130:44453/ using document root <none> and password file <none>
I20260811 00:22:25.669765 6581 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:25.669994 6581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:25.670472 6581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:22:25 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407745, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:25.715180 6581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "cf15eaef4d704df8b99e271b9bd6122e"
format_stamp: "Formatted at 2026-08-11 00:22:25 on dist-test-slave-f7th"
server_key: "819d3f428f052f19c5780528a58f9517"
server_key_iv: "325e3baf33ef45aa7f2cd0a53b772d07"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:25.716734 6581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "cf15eaef4d704df8b99e271b9bd6122e"
format_stamp: "Formatted at 2026-08-11 00:22:25 on dist-test-slave-f7th"
server_key: "819d3f428f052f19c5780528a58f9517"
server_key_iv: "325e3baf33ef45aa7f2cd0a53b772d07"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:25.733264 6581 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.006s sys 0.007s
I20260811 00:22:25.776077 6619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:25.779800 6581 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.006s
I20260811 00:22:25.780198 6581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "cf15eaef4d704df8b99e271b9bd6122e"
format_stamp: "Formatted at 2026-08-11 00:22:25 on dist-test-slave-f7th"
server_key: "819d3f428f052f19c5780528a58f9517"
server_key_iv: "325e3baf33ef45aa7f2cd0a53b772d07"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:25.780606 6581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:25.854755 6581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:25.861316 6581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:25.861835 6581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:25.869967 6581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:25.879133 6581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:25.879377 6581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:22:25.879585 6581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:25.879705 6581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:25.911743 6571 heartbeater.cc:503] Master 127.16.71.190:34273 was elected leader, sending a full tablet report...
I20260811 00:22:26.063688 6581 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:35883
I20260811 00:22:26.063732 6833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:35883 every 8 connection(s)
I20260811 00:22:26.065790 6581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:22:26.072824 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 6581
I20260811 00:22:26.073045 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:22:26.073746 16670 external_mini_cluster.cc:1428] Setting key abb71568a52f0533ef522f028fa5bf3d
Aug 11 00:22:26 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407745, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
I20260811 00:22:26.116770 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:26.069103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:34369 (local address 127.16.71.190:34273)
0811 00:22:26.087868 (+ 18765us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:26.087875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:26.087911 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:22:26.091424 (+ 3513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:26.091431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:26.091550 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:26.091688 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:26.094039 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.095285 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.098027 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.098682 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.098789 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:26.103816 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:26.103823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:26.103828 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:26.103880 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:26.107542 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.112919 (+ 5377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.112927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.112931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.113169 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.113954 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.113960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.113963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.114382 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:26.114576 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:26.115951 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0811 00:22:26.116229 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18593,"thread_start_us":18806,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
I20260811 00:22:26.118750 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:26.070012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34273 (local address 127.16.71.130:34369)
0811 00:22:26.087155 (+ 17143us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:26.087242 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:26.090998 (+ 3756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:26.092015 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:26.092031 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:26.092774 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:26.093683 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:26.093712 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.095579 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.095585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:26.097799 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:26.097820 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.099166 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.099173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:26.099506 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:26.100501 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:26.100534 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:26.103487 (+ 2953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:26.107810 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:26.107821 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:26.107842 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:26.108493 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:26.113363 (+ 4870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:26.113369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:26.113373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:26.113747 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:26.114571 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:26.114581 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:26.115123 (+ 542us) client_negotiation.cc:769] Sending connection context
0811 00:22:26.115784 (+ 661us) client_negotiation.cc:240] Negotiation successful
0811 00:22:26.116422 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16694,"thread_start_us":17693,"threads_started":1}
I20260811 00:22:26.122125 6834 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34273
I20260811 00:22:26.122530 6834 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:26.123512 6834 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
I20260811 00:22:26.125393 6170 ts_manager.cc:194] Registered new tserver with Master: cf15eaef4d704df8b99e271b9bd6122e (127.16.71.130:35883)
I20260811 00:22:26.126572 6170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:34369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:22:26.162987 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34273
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:26.564518 6845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:26.564929 6845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:26.565090 6845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:26.573695 6845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:26.573906 6845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:26.574098 6845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:22:26.584990 6845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.71.148:49053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:26.586567 6845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:26.588100 6845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:26.603282 6856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:26.603370 6854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:26.603487 6858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:26.604820 6845 server_base.cc:1034] running on GCE node
I20260811 00:22:26.605660 6845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:26.607127 6845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:26.608408 6845 hybrid_clock.cc:648] HybridClock initialized: now 1786407746608350 us; error 47 us; skew 500 ppm
I20260811 00:22:26.608846 6845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:26 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407746, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:26.612808 6845 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:22:26.614634 6845 webserver.cc:460] Webserver started at http://127.16.71.131:44571/ using document root <none> and password file <none>
I20260811 00:22:26.615446 6845 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:26.615586 6845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:26.615936 6845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:22:26 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407746, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:26.637985 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "be87ac007c8a4145bbb3a0ec929444bf"
format_stamp: "Formatted at 2026-08-11 00:22:26 on dist-test-slave-f7th"
server_key: "0a87790e9e5a4c95d7208a6f8670dec0"
server_key_iv: "5c0ea78378711b1eb363918634e9fa27"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:26.639461 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "be87ac007c8a4145bbb3a0ec929444bf"
format_stamp: "Formatted at 2026-08-11 00:22:26 on dist-test-slave-f7th"
server_key: "0a87790e9e5a4c95d7208a6f8670dec0"
server_key_iv: "5c0ea78378711b1eb363918634e9fa27"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:26.648332 6845 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260811 00:22:26.666043 6873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:26.667793 6845 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:22:26.668080 6845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "be87ac007c8a4145bbb3a0ec929444bf"
format_stamp: "Formatted at 2026-08-11 00:22:26 on dist-test-slave-f7th"
server_key: "0a87790e9e5a4c95d7208a6f8670dec0"
server_key_iv: "5c0ea78378711b1eb363918634e9fa27"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:26.668337 6845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:26.686992 6845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:26.691828 6845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:26.692180 6845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:26.693370 6845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:26.694985 6845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:26.695079 6845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:26.695216 6845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:26.695273 6845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:26.730188 6845 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:46583
I20260811 00:22:26.730233 7092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:46583 every 8 connection(s)
I20260811 00:22:26.731701 6845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:22:26.735174 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 6845
I20260811 00:22:26.735390 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:22:26.735909 16670 external_mini_cluster.cc:1428] Setting key 20ad5324b47066bffd0aa045ac5af4ea
Aug 11 00:22:26 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407746, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:26.755242 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:26.734855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:34069 (local address 127.16.71.190:34273)
0811 00:22:26.735499 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:26.735506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:26.736750 (+ 1244us) server_negotiation.cc:407] Connection header received
0811 00:22:26.739192 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:26.739198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:26.739336 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:26.739441 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:26.741265 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.742106 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.744006 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.744505 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.744659 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:26.748553 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:26.748559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:26.748563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:26.748610 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:26.751548 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.752485 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.752490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.752492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.752674 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.753193 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.753197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.753199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.753526 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:26.753678 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:26.754566 (+ 888us) server_negotiation.cc:299] Negotiation successful
0811 00:22:26.754853 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":339,"threads_started":1}
I20260811 00:22:26.756362 7097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:26.735443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34273 (local address 127.16.71.131:34069)
0811 00:22:26.736504 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:26.736570 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:26.738863 (+ 2293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:26.739746 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:26.739757 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:26.740278 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:26.741053 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:26.741077 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.742317 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.742322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:26.743794 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:26.743813 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.744838 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.744844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:26.745091 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:26.745861 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:26.745888 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:26.748199 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:26.751748 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:26.751757 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:26.751776 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:26.752332 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:26.752801 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:26.752805 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:26.752807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:26.753063 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:26.753679 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:26.753689 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:26.754100 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 00:22:26.754429 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 00:22:26.755088 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":381,"threads_started":1}
I20260811 00:22:26.758163 7094 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34273
I20260811 00:22:26.758478 7094 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:26.759248 7094 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
I20260811 00:22:26.761034 6170 ts_manager.cc:194] Registered new tserver with Master: be87ac007c8a4145bbb3a0ec929444bf (127.16.71.131:46583)
I20260811 00:22:26.761951 6170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:34069
I20260811 00:22:26.765715 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:22:26.767686 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 6320
Aug 11 00:22:26 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:22:26.787292 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:26.768915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.16.71.129:37467)
0811 00:22:26.769634 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:26.769642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:26.769670 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:22:26.769733 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:26.769740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:26.769962 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:26.770115 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:26.771867 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.773140 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.774944 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.775615 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.775746 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:26.780725 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:26.780745 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:26.780764 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:26.780822 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:26.783942 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.784937 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.784946 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.784949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.785168 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.785626 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.785631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.785633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.786056 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:26.786324 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:26.786659 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 00:22:26.786922 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":385,"threads_started":1}
I20260811 00:22:26.788758 6564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:26.876000 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 6320
I20260811 00:22:26.907191 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 6581
Aug 11 00:22:26 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:22:26.929834 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:26.912743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.16.71.130:35883)
0811 00:22:26.913322 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:26.913330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:26.913357 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:22:26.913432 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:26.913439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:26.913685 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:26.913828 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:26.915144 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.916185 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.917706 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:26.918302 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:26.918415 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:26.923242 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:26.923259 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:26.923276 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:26.923321 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:26.926592 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.927507 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.927515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.927518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.927699 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:26.928134 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:26.928139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:26.928141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:26.928558 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:26.928858 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:26.929190 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 00:22:26.929425 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":308,"threads_started":1}
I20260811 00:22:26.931347 6828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:27.019357 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 6581
I20260811 00:22:27.052990 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 6845
Aug 11 00:22:27 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:22:27.073060 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:27.054164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59028 (local address 127.16.71.131:46583)
0811 00:22:27.054878 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:27.054887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:27.054917 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:22:27.055035 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:27.055042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:27.055288 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:27.055474 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:27.057073 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:27.058184 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:27.059710 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:27.060347 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:27.060490 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:27.065985 (+ 5495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:27.066008 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:27.066028 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:27.066087 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:27.069615 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:27.070496 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:27.070505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:27.070508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:27.070678 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:27.071186 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:27.071190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:27.071192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:27.071621 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:27.071897 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:27.072341 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 00:22:27.072599 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":378,"threads_started":1}
I20260811 00:22:27.074690 7083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:27.159865 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 6845
I20260811 00:22:27.184551 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 6105
I20260811 00:22:27.186149 6224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:27.298863 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 6105
I20260811 00:22:27.335088 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34273
--webserver_interface=127.16.71.190
--webserver_port=42767
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:34273
--ranger_config_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:27.701951 7112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:27.702281 7112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:27.702375 7112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:27.709941 7112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:27.710086 7112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:27.710135 7112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:27.710197 7112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:27.710248 7112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:27.721101 7112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:34273
--ranger_config_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=42767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:27.722519 7112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:27.723968 7112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:27.737020 7125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:27.737097 7122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:27.737068 7127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:27.738152 7112 server_base.cc:1034] running on GCE node
I20260811 00:22:27.739074 7112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:27.740708 7112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:27.741961 7112 hybrid_clock.cc:648] HybridClock initialized: now 1786407747741918 us; error 30 us; skew 500 ppm
I20260811 00:22:27.742386 7112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:27 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407747, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:27.749265 7112 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:22:27.751129 7112 webserver.cc:460] Webserver started at http://127.16.71.190:42767/ using document root <none> and password file <none>
I20260811 00:22:27.751883 7112 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:27.752024 7112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:22:27 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407747, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:27.838564 7112 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.001s
I20260811 00:22:27.843410 7143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:27.845669 7112 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 00:22:27.845948 7112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "2d358568f8cd47b1832ad035aea71b0b"
format_stamp: "Formatted at 2026-08-11 00:22:22 on dist-test-slave-f7th"
server_key: "843ba82369b3924c641f734a709fb257"
server_key_iv: "fe5aaff08b32db8bec13b202de72450d"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:27.847115 7112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:27.893006 7112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:27.897756 7112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:27.898063 7112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:27.918432 7112 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:34273
I20260811 00:22:27.918478 7242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:34273 every 8 connection(s)
I20260811 00:22:27.920146 7112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:22:27.928458 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 7112
I20260811 00:22:27.928956 7245 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:22:27.929702 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:37467
--local_ip_for_outbound_sockets=127.16.71.129
--tserver_master_addrs=127.16.71.190:34273
--webserver_port=37893
--webserver_interface=127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:22:27.933568 7245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Bootstrap starting.
I20260811 00:22:27.959126 7245 log.cc:824] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:27.973645 7245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:22:27.974556 7245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Bootstrap complete.
I20260811 00:22:27.990144 7245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:27.991722 7245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d358568f8cd47b1832ad035aea71b0b, State: Initialized, Role: FOLLOWER
I20260811 00:22:27.992801 7245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:27.993429 7245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:22:27.993608 7245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:22:27.993831 7245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:22:28.000474 7245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:28.001336 7245 leader_election.cc:302] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d358568f8cd47b1832ad035aea71b0b; no voters:
I20260811 00:22:28.002154 7245 leader_election.cc:288] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:22:28.002345 7250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:22:28.003685 7250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [term 2 LEADER]: Becoming Leader. State: Replica: 2d358568f8cd47b1832ad035aea71b0b, State: Running, Role: LEADER
I20260811 00:22:28.004341 7250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } }
I20260811 00:22:28.004817 7245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:22:28.007156 7251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2d358568f8cd47b1832ad035aea71b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } } }
I20260811 00:22:28.007586 7251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:28.008313 7254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d358568f8cd47b1832ad035aea71b0b. Latest consensus state: current_term: 2 leader_uuid: "2d358568f8cd47b1832ad035aea71b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d358568f8cd47b1832ad035aea71b0b" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 34273 } } }
I20260811 00:22:28.008790 7254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:28.018610 7259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:22:28.025381 7259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:22:28.027231 7259 catalog_manager.cc:1259] Loaded cluster ID: a765bbe76a0348b59e1903e2066a4c34
I20260811 00:22:28.027467 7259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:22:28.032452 7259 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:22:28.034240 7259 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Loaded TSK: 0
I20260811 00:22:28.036028 7259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:22:28.038264 7245 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 00:22:28.458935 7247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:28.459501 7247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:28.459714 7247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:28.472250 7247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:28.472514 7247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:28.472827 7247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:22:28.489843 7247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:37467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=37893
--tserver_master_addrs=127.16.71.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:28.492105 7247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:28.494231 7247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:28.517124 7320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:28.519548 7325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:28.530421 7327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:28.530821 7247 server_base.cc:1034] running on GCE node
I20260811 00:22:28.531790 7247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:28.533447 7247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:28.534765 7247 hybrid_clock.cc:648] HybridClock initialized: now 1786407748534678 us; error 75 us; skew 500 ppm
I20260811 00:22:28.535266 7247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:28 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407748, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:28.540558 7247 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:22:28.543102 7247 webserver.cc:460] Webserver started at http://127.16.71.129:37893/ using document root <none> and password file <none>
I20260811 00:22:28.544337 7247 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:28.544564 7247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:22:28 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407748, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:28.581883 7247 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.005s sys 0.000s
I20260811 00:22:28.585954 7345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:28.587690 7247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:22:28.587910 7247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "ad629f3b7117495580576d9b13c84789"
format_stamp: "Formatted at 2026-08-11 00:22:24 on dist-test-slave-f7th"
server_key: "3bab14e672461704c2a19ab75af6c80e"
server_key_iv: "d44509802758bad577f193b9ee7b7b2e"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:28.589579 7247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:28.622747 7247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:28.627561 7247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:28.627895 7247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:28.629231 7247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:28.633571 7247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:28.633704 7247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:28.633828 7247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:28.633909 7247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:28.688050 7247 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:37467
I20260811 00:22:28.688097 7556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:37467 every 8 connection(s)
I20260811 00:22:28.689731 7247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:22:28.697379 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 7247
I20260811 00:22:28.698293 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:35883
--local_ip_for_outbound_sockets=127.16.71.130
--tserver_master_addrs=127.16.71.190:34273
--webserver_port=44453
--webserver_interface=127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:22:28 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407748, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:28.719633 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:28.693330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:34311 (local address 127.16.71.190:34273)
0811 00:22:28.695049 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:28.695069 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:28.695126 (+ 57us) server_negotiation.cc:407] Connection header received
0811 00:22:28.697308 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:28.697322 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:28.697761 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:28.698143 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:28.700070 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:28.701497 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:28.703193 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:28.704192 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:28.704401 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:28.708787 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:28.708797 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:28.708814 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:28.708869 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:28.712819 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:28.714094 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:28.714104 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:28.714113 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:28.714379 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:28.715068 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:28.715073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:28.715077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:28.715909 (+ 832us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:28.716138 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:28.717524 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0811 00:22:28.718006 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":1198,"threads_started":1}
I20260811 00:22:28.725945 7561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:28.693318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34273 (local address 127.16.71.129:34311)
0811 00:22:28.694171 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:28.694228 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:28.696438 (+ 2210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:28.698336 (+ 1898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:28.698360 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:28.699168 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:28.699809 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:28.699830 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:28.701692 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:28.701696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:28.702992 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:28.703025 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:28.704823 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:28.704829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:28.705041 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:28.705810 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:28.705848 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:28.708417 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:28.713071 (+ 4654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:28.713082 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:28.713101 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:28.713835 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:28.714536 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:28.714542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:28.714546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:28.714868 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:28.716115 (+ 1247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:28.716126 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:28.716681 (+ 555us) client_negotiation.cc:769] Sending connection context
0811 00:22:28.717218 (+ 537us) client_negotiation.cc:240] Negotiation successful
0811 00:22:28.717951 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":333,"threads_started":1}
I20260811 00:22:28.730429 7559 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34273
I20260811 00:22:28.730880 7559 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:28.732048 7559 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
W20260811 00:22:28.736773 7559 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:34273 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:22:28.737118 7559 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:28.737485 7559 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
I20260811 00:22:28.739117 7559 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:28.739500 7559 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
W20260811 00:22:28.740906 7559 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:22:29.137638 7563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:29.137992 7563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:29.138154 7563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:29.150421 7563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:29.150604 7563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:29.150785 7563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:22:29.162650 7563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:35883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=44453
--tserver_master_addrs=127.16.71.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:29.164572 7563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:29.166626 7563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:29.183535 7596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:29.184716 7593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:29.185901 7598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:29.187145 7563 server_base.cc:1034] running on GCE node
I20260811 00:22:29.188009 7563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:29.189394 7563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:29.190752 7563 hybrid_clock.cc:648] HybridClock initialized: now 1786407749190662 us; error 68 us; skew 500 ppm
I20260811 00:22:29.191186 7563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:29 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:29.194792 7563 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:22:29.197059 7563 webserver.cc:460] Webserver started at http://127.16.71.130:44453/ using document root <none> and password file <none>
I20260811 00:22:29.198024 7563 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:29.198231 7563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:22:29 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:29.233111 7563 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.006s sys 0.005s
I20260811 00:22:29.249433 7611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:29.252017 7563 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.007s sys 0.008s
I20260811 00:22:29.252281 7563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "cf15eaef4d704df8b99e271b9bd6122e"
format_stamp: "Formatted at 2026-08-11 00:22:25 on dist-test-slave-f7th"
server_key: "819d3f428f052f19c5780528a58f9517"
server_key_iv: "325e3baf33ef45aa7f2cd0a53b772d07"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:29.253934 7563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:29.279811 7563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:29.286031 7563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:29.286410 7563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:29.301095 7563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:29.304451 7563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:29.304651 7563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:29.304792 7563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:29.304894 7563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:22:29 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:29.370779 7563 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:35883
I20260811 00:22:29.370844 7817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:35883 every 8 connection(s)
I20260811 00:22:29.372977 7563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:22:29.380940 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 7563
I20260811 00:22:29.381837 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.71.148:49053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:46583
--local_ip_for_outbound_sockets=127.16.71.131
--tserver_master_addrs=127.16.71.190:34273
--webserver_port=44571
--webserver_interface=127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:22:29 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:29.419389 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:29.377967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:33197 (local address 127.16.71.190:34273)
0811 00:22:29.388247 (+ 10280us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:29.388254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:29.388279 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:22:29.388351 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:29.388356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:29.388452 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:29.388605 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:29.398206 (+ 9601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:29.399459 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:29.402471 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:29.403103 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:29.403217 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:29.409913 (+ 6696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:29.409922 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:29.409928 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:29.409977 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:29.413974 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:29.415288 (+ 1314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:29.415295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:29.415299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:29.415489 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:29.416187 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:29.416194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:29.416197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:29.416596 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:29.416794 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:29.418652 (+ 1858us) server_negotiation.cc:299] Negotiation successful
0811 00:22:29.418972 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10122,"thread_start_us":14673,"threads_started":1}
I20260811 00:22:29.422356 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:29.377214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34273 (local address 127.16.71.130:33197)
0811 00:22:29.385285 (+ 8071us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:29.385341 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:29.387237 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:29.395856 (+ 8619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:29.395876 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:29.396822 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:29.397867 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:29.397904 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:29.399762 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:29.399768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:29.402182 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:29.402217 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:29.403657 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:29.403664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:29.403978 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:29.404847 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:29.404881 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:29.409539 (+ 4658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:29.414251 (+ 4712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:29.414263 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:29.414288 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:29.415087 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:29.415673 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:29.415678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:29.415682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:29.416031 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:29.416917 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:29.416925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:29.417455 (+ 530us) client_negotiation.cc:769] Sending connection context
0811 00:22:29.420045 (+ 2590us) client_negotiation.cc:240] Negotiation successful
0811 00:22:29.420774 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7703,"spinlock_wait_cycles":30080,"thread_start_us":7604,"threads_started":1}
I20260811 00:22:29.425450 7818 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34273
I20260811 00:22:29.425832 7818 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:29.426644 7818 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
I20260811 00:22:29.428766 7174 ts_manager.cc:194] Registered new tserver with Master: cf15eaef4d704df8b99e271b9bd6122e (127.16.71.130:35883)
I20260811 00:22:29.430559 7174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:33197
I20260811 00:22:29.741074 7559 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:29.741592 7559 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
I20260811 00:22:29.743031 7174 ts_manager.cc:194] Registered new tserver with Master: ad629f3b7117495580576d9b13c84789 (127.16.71.129:37467)
I20260811 00:22:29.744273 7174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:34311
Aug 11 00:22:29 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
W20260811 00:22:29.927167 7824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:29.927726 7824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:29.927908 7824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:29.939224 7824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:29.939522 7824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:29.939772 7824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:22:29.955070 7824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.71.148:49053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:46583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=44571
--tserver_master_addrs=127.16.71.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:29.956943 7824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:29.958773 7824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:29.977609 7844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:29.979633 7847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:29.987375 7854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:29.988406 7824 server_base.cc:1034] running on GCE node
I20260811 00:22:29.989396 7824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:29.990808 7824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:29.992091 7824 hybrid_clock.cc:648] HybridClock initialized: now 1786407749992020 us; error 59 us; skew 500 ppm
I20260811 00:22:29.992513 7824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:29 dist-test-slave-f7th krb5kdc[5283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:29.997182 7824 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:22:30.001691 7824 webserver.cc:460] Webserver started at http://127.16.71.131:44571/ using document root <none> and password file <none>
I20260811 00:22:30.002476 7824 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:30.002615 7824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:22:30 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:30.042270 7824 fs_manager.cc:709] Time spent opening directory manager: real 0.022s user 0.006s sys 0.007s
I20260811 00:22:30.045581 7868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:30.047254 7824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:22:30.047454 7824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "be87ac007c8a4145bbb3a0ec929444bf"
format_stamp: "Formatted at 2026-08-11 00:22:26 on dist-test-slave-f7th"
server_key: "0a87790e9e5a4c95d7208a6f8670dec0"
server_key_iv: "5c0ea78378711b1eb363918634e9fa27"
server_key_version: "kuduclusterkey@0"
I20260811 00:22:30.048556 7824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:30.071798 7824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:30.076555 7824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:30.076915 7824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:30.078116 7824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:30.079901 7824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:30.080032 7824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:30.080160 7824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:30.080236 7824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:30.124357 7824 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:46583
I20260811 00:22:30.124423 8097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:46583 every 8 connection(s)
I20260811 00:22:30.126093 7824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:22:30.134703 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 7824
Aug 11 00:22:30 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:30.175513 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.135978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.16.71.190:34273)
0811 00:22:30.155271 (+ 19293us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.155277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.155299 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:22:30.155352 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.155357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.155485 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.155610 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:30.158019 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.159173 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.161162 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.161984 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.162100 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.166603 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:30.166610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:30.166613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:30.166683 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:30.169589 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.171322 (+ 1733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.171328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.171332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.171521 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.172780 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.172785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.172788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.173205 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:30.173552 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.174875 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.175123 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19155,"thread_start_us":17867,"threads_started":1}
I20260811 00:22:30.177726 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.132021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:45189 (local address 127.16.71.190:34273)
0811 00:22:30.154290 (+ 22269us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.154296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.154321 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:22:30.154409 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.154415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.154517 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.154645 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:30.157237 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.158354 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.160985 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.161631 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.161750 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.166515 (+ 4765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:30.166523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:30.166528 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:30.166600 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:30.172175 (+ 5575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.173284 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.173289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.173291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.173453 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.174102 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.174107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.174110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.174439 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:30.174604 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.176577 (+ 1973us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.177309 (+ 732us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22117,"thread_start_us":22871,"threads_started":1}
I20260811 00:22:30.179538 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.131257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:34273 (local address 127.16.71.131:45189)
0811 00:22:30.149505 (+ 18248us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.149587 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:30.152523 (+ 2936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:30.155070 (+ 2547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:30.155085 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:30.155906 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:30.156957 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.156994 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.158574 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.158581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.160731 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.160768 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.162136 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.162144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.162477 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.163351 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:30.163395 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:30.166223 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:30.172353 (+ 6130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:30.172365 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:30.172387 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:30.173127 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:30.173610 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:30.173615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:30.173618 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:30.173958 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:30.174848 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:30.174857 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:30.175360 (+ 503us) client_negotiation.cc:769] Sending connection context
0811 00:22:30.175695 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 00:22:30.176834 (+ 1139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17786,"thread_start_us":17458,"threads_started":1}
I20260811 00:22:30.181152 8098 heartbeater.cc:347] Connected to a master server at 127.16.71.190:34273
I20260811 00:22:30.181535 8098 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:30.182560 8098 heartbeater.cc:511] Master 127.16.71.190:34273 requested a full tablet report, sending...
I20260811 00:22:30.184695 7174 ts_manager.cc:194] Registered new tserver with Master: be87ac007c8a4145bbb3a0ec929444bf (127.16.71.131:46583)
I20260811 00:22:30.185588 7174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:45189
I20260811 00:22:30.191470 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:22:30.233752 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.215476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.16.71.190:34273)
0811 00:22:30.215678 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.215683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.217258 (+ 1575us) server_negotiation.cc:407] Connection header received
0811 00:22:30.217446 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.217451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.217542 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.217633 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:30.219469 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.220668 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.222300 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.222739 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.222866 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.226416 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:30.226425 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:30.226430 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:30.226498 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:30.230233 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.231409 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.231416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.231419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.231630 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.232093 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.232099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.232102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.232434 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:30.232724 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.233150 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.233409 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 00:22:30 dist-test-slave-f7th krb5kdc[5283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for HTTP/127.16.71.148@KRBTEST.COM
I20260811 00:22:30.239964 7174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:22:30.251295 7174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:22:30.336882 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.304249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:46583 (local address 127.0.0.1:59030)
0811 00:22:30.322374 (+ 18125us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.322397 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:30.323376 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:30.325300 (+ 1924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:30.325307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:30.325365 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:30.325869 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.325882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.329331 (+ 3449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.329338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.331998 (+ 2660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.332032 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.335430 (+ 3398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.335439 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.335747 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.335833 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 00:22:30.336041 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 00:22:30.336311 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17964,"thread_start_us":14222,"threads_started":1}
I20260811 00:22:30.338734 8131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.304984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59030 (local address 127.16.71.131:46583)
0811 00:22:30.317249 (+ 12265us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.317260 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.322576 (+ 5316us) server_negotiation.cc:407] Connection header received
0811 00:22:30.323917 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.323930 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.324294 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.324555 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:30.327218 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.329033 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.332349 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.334656 (+ 2307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.334856 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.335174 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.337644 (+ 2470us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.338148 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12071,"thread_start_us":11886,"threads_started":1}
I20260811 00:22:30.344484 7960 tablet_service.cc:1467] Processing CreateTablet for tablet a08c33c9bd22489db284b79d6c954c35 (DEFAULT_TABLE table=test-table [id=d9d538ca836f4de481d732968528b3f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:30.348107 7960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08c33c9bd22489db284b79d6c954c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:30.348376 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.312118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.16.71.129:37467)
0811 00:22:30.326846 (+ 14728us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.326857 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.337863 (+ 11006us) server_negotiation.cc:407] Connection header received
0811 00:22:30.338329 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.338340 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.338672 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.338924 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:30.341017 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.342758 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.345406 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.346862 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.346965 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.347178 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.347815 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.348023 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14480,"thread_start_us":14729,"threads_started":1}
I20260811 00:22:30.349243 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.311362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:44910)
0811 00:22:30.337704 (+ 26342us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.337728 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:30.338077 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:30.340484 (+ 2407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:30.340490 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:30.340526 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:30.340757 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.340771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.343009 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.343015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.345208 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.345230 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.347341 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.347346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.347583 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.347622 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:22:30.348707 (+ 1085us) client_negotiation.cc:240] Negotiation successful
0811 00:22:30.348868 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26203,"mutex_wait_us":20,"thread_start_us":10653,"threads_started":1}
I20260811 00:22:30.349339 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.303842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:35883 (local address 127.0.0.1:60550)
0811 00:22:30.318442 (+ 14600us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.318517 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:30.318784 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:30.321030 (+ 2246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:30.321040 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:30.321087 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:30.322646 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.322666 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.326717 (+ 4051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.326725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.329596 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.329624 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.347334 (+ 17710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.347342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.348753 (+ 1411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.348788 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:22:30.348872 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 00:22:30.348994 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14252,"mutex_wait_us":1156,"thread_start_us":40924,"threads_started":1}
I20260811 00:22:30.350133 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.304519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.16.71.130:35883)
0811 00:22:30.317719 (+ 13200us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.317729 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.319091 (+ 1362us) server_negotiation.cc:407] Connection header received
0811 00:22:30.319215 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.319223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.319517 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.319746 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:30.322951 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.324795 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.345119 (+ 20324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.346717 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.346907 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.347139 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.349610 (+ 2471us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.349813 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13016,"thread_start_us":12907,"threads_started":1}
I20260811 00:22:30.353672 7704 tablet_service.cc:1467] Processing CreateTablet for tablet a08c33c9bd22489db284b79d6c954c35 (DEFAULT_TABLE table=test-table [id=d9d538ca836f4de481d732968528b3f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:30.355213 7704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08c33c9bd22489db284b79d6c954c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:30.353672 7433 tablet_service.cc:1467] Processing CreateTablet for tablet a08c33c9bd22489db284b79d6c954c35 (DEFAULT_TABLE table=test-table [id=d9d538ca836f4de481d732968528b3f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:30.356539 7433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08c33c9bd22489db284b79d6c954c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:30.373962 8136 tablet_bootstrap.cc:492] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Bootstrap starting.
I20260811 00:22:30.383636 8139 tablet_bootstrap.cc:492] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Bootstrap starting.
I20260811 00:22:30.384821 8140 tablet_bootstrap.cc:492] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Bootstrap starting.
I20260811 00:22:30.387858 8139 tablet_bootstrap.cc:654] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:30.389168 8140 tablet_bootstrap.cc:654] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:30.389374 8139 log.cc:824] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:30.390159 8136 tablet_bootstrap.cc:654] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:30.390551 8140 log.cc:824] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:30.391645 8136 log.cc:824] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:30.396672 8139 tablet_bootstrap.cc:492] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: No bootstrap required, opened a new log
I20260811 00:22:30.397066 8139 ts_tablet_manager.cc:1397] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20260811 00:22:30.399252 8140 tablet_bootstrap.cc:492] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: No bootstrap required, opened a new log
I20260811 00:22:30.399619 8140 ts_tablet_manager.cc:1397] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.005s
I20260811 00:22:30.406329 8136 tablet_bootstrap.cc:492] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: No bootstrap required, opened a new log
I20260811 00:22:30.406670 8136 ts_tablet_manager.cc:1397] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.005s
I20260811 00:22:30.410542 8140 raft_consensus.cc:348] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.411129 8140 raft_consensus.cc:374] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:30.411254 8140 raft_consensus.cc:729] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad629f3b7117495580576d9b13c84789, State: Initialized, Role: FOLLOWER
I20260811 00:22:30.412173 8140 consensus_queue.cc:260] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.415766 8136 raft_consensus.cc:348] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.416338 8136 raft_consensus.cc:374] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:30.416461 8136 raft_consensus.cc:729] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be87ac007c8a4145bbb3a0ec929444bf, State: Initialized, Role: FOLLOWER
I20260811 00:22:30.417390 8136 consensus_queue.cc:260] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.420344 7559 heartbeater.cc:503] Master 127.16.71.190:34273 was elected leader, sending a full tablet report...
I20260811 00:22:30.419941 8139 raft_consensus.cc:348] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.420661 8140 ts_tablet_manager.cc:1428] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Time spent starting tablet: real 0.021s user 0.017s sys 0.004s
I20260811 00:22:30.420832 8139 raft_consensus.cc:374] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:30.421021 8139 raft_consensus.cc:729] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf15eaef4d704df8b99e271b9bd6122e, State: Initialized, Role: FOLLOWER
I20260811 00:22:30.422163 8139 consensus_queue.cc:260] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.423173 8147 raft_consensus.cc:482] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:22:30.423466 8147 raft_consensus.cc:504] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.425762 8147 leader_election.cc:288] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf15eaef4d704df8b99e271b9bd6122e (127.16.71.130:35883), be87ac007c8a4145bbb3a0ec929444bf (127.16.71.131:46583)
I20260811 00:22:30.428324 8136 ts_tablet_manager.cc:1428] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Time spent starting tablet: real 0.022s user 0.016s sys 0.000s
I20260811 00:22:30.429821 8098 heartbeater.cc:503] Master 127.16.71.190:34273 was elected leader, sending a full tablet report...
I20260811 00:22:30.432717 8139 ts_tablet_manager.cc:1428] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Time spent starting tablet: real 0.036s user 0.017s sys 0.005s
I20260811 00:22:30.433317 7818 heartbeater.cc:503] Master 127.16.71.190:34273 was elected leader, sending a full tablet report...
W20260811 00:22:30.444768 7560 tablet.cc:2369] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:22:30.456836 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.427017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:35669 (local address 127.16.71.130:35883)
0811 00:22:30.427250 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.427256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.437287 (+ 10031us) server_negotiation.cc:407] Connection header received
0811 00:22:30.437394 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.437399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.437479 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.437565 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:30.441273 (+ 3708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.442426 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.451771 (+ 9345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.453730 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.453945 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.454038 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.456079 (+ 2041us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.456284 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:22:30.459414 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.426177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:35883 (local address 127.16.71.129:35669)
0811 00:22:30.436167 (+ 9990us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.436198 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:30.436398 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:30.440214 (+ 3816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:30.440220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:30.440255 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:30.440933 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.441025 (+ 92us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.442631 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.442638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.451398 (+ 8760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.451423 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.455065 (+ 3642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.455076 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.455324 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.455369 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:22:30.458700 (+ 3331us) client_negotiation.cc:240] Negotiation successful
0811 00:22:30.458928 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9744,"thread_start_us":10635,"threads_started":1}
I20260811 00:22:30.461520 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08c33c9bd22489db284b79d6c954c35" candidate_uuid: "ad629f3b7117495580576d9b13c84789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf15eaef4d704df8b99e271b9bd6122e" is_pre_election: true
I20260811 00:22:30.462268 7736 raft_consensus.cc:2454] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad629f3b7117495580576d9b13c84789 in term 0.
I20260811 00:22:30.463565 7347 leader_election.cc:302] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad629f3b7117495580576d9b13c84789, cf15eaef4d704df8b99e271b9bd6122e; no voters:
I20260811 00:22:30.464221 8131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.426388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:53095 (local address 127.16.71.131:46583)
0811 00:22:30.426835 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.426841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.437724 (+ 10883us) server_negotiation.cc:407] Connection header received
0811 00:22:30.437807 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.437812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.437889 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.437980 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:30.448171 (+ 10191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.449419 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.461116 (+ 11697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.462423 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.462755 (+ 332us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.462847 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.463745 (+ 898us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.463940 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260811 00:22:30.464571 8147 raft_consensus.cc:2781] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:22:30.464808 8147 raft_consensus.cc:482] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:22:30.464946 8147 raft_consensus.cc:3037] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:30.468494 8147 raft_consensus.cc:504] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.470817 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08c33c9bd22489db284b79d6c954c35" candidate_uuid: "ad629f3b7117495580576d9b13c84789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf15eaef4d704df8b99e271b9bd6122e"
I20260811 00:22:30.470963 8147 leader_election.cc:288] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [CANDIDATE]: Term 1 election: Requested vote from peers cf15eaef4d704df8b99e271b9bd6122e (127.16.71.130:35883), be87ac007c8a4145bbb3a0ec929444bf (127.16.71.131:46583)
I20260811 00:22:30.471196 7736 raft_consensus.cc:3037] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:30.471756 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.426350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:46583 (local address 127.16.71.129:53095)
0811 00:22:30.437150 (+ 10800us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.437171 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:30.437318 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:30.442897 (+ 5579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:30.442902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:30.442944 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:30.447801 (+ 4857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.447825 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.449721 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.449729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.460752 (+ 11023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:30.460786 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.463232 (+ 2446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.463252 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:30.463482 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.463525 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 00:22:30.471145 (+ 7620us) client_negotiation.cc:240] Negotiation successful
0811 00:22:30.471340 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10668,"thread_start_us":14654,"threads_started":1}
I20260811 00:22:30.473346 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08c33c9bd22489db284b79d6c954c35" candidate_uuid: "ad629f3b7117495580576d9b13c84789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be87ac007c8a4145bbb3a0ec929444bf" is_pre_election: true
I20260811 00:22:30.474059 8006 raft_consensus.cc:2454] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad629f3b7117495580576d9b13c84789 in term 0.
I20260811 00:22:30.474715 8000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08c33c9bd22489db284b79d6c954c35" candidate_uuid: "ad629f3b7117495580576d9b13c84789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be87ac007c8a4145bbb3a0ec929444bf"
I20260811 00:22:30.474833 7736 raft_consensus.cc:2454] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad629f3b7117495580576d9b13c84789 in term 1.
I20260811 00:22:30.474990 8000 raft_consensus.cc:3037] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:30.478794 8000 raft_consensus.cc:2454] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad629f3b7117495580576d9b13c84789 in term 1.
I20260811 00:22:30.479490 7347 leader_election.cc:302] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad629f3b7117495580576d9b13c84789, cf15eaef4d704df8b99e271b9bd6122e; no voters:
I20260811 00:22:30.479954 8147 raft_consensus.cc:2781] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:22:30.489063 8147 raft_consensus.cc:686] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 1 LEADER]: Becoming Leader. State: Replica: ad629f3b7117495580576d9b13c84789, State: Running, Role: LEADER
I20260811 00:22:30.489814 8147 consensus_queue.cc:237] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } }
I20260811 00:22:30.496440 7174 catalog_manager.cc:5495] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad629f3b7117495580576d9b13c84789 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "ad629f3b7117495580576d9b13c84789" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad629f3b7117495580576d9b13c84789" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 37467 } health_report { overall_health: HEALTHY } } }
I20260811 00:22:30.549597 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.541330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.16.71.129:37467)
0811 00:22:30.541558 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.541565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.541591 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:22:30.541926 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.541932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.542083 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.542290 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:22:30.542996 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.544134 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.546405 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.547073 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.547209 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.547952 (+ 743us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:22:30.548819 (+ 867us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:22:30.548982 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.549162 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.549267 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:22:30.560998 8000 raft_consensus.cc:1261] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 1 FOLLOWER]: Refusing update from remote peer ad629f3b7117495580576d9b13c84789: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:30.561044 7736 raft_consensus.cc:1261] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 1 FOLLOWER]: Refusing update from remote peer ad629f3b7117495580576d9b13c84789: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:30.562250 8147 consensus_queue.cc:1035] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf15eaef4d704df8b99e271b9bd6122e" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 35883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:22:30.562654 8157 consensus_queue.cc:1035] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be87ac007c8a4145bbb3a0ec929444bf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 46583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:22:30.600111 8161 mvcc.cc:205] Tried to move back new op lower bound from 7317126146284765184 to 7317126146011922432. Current Snapshot: MvccSnapshot[applied={T|T < 7317126146284765184}]
I20260811 00:22:30.610018 8171 mvcc.cc:205] Tried to move back new op lower bound from 7317126146284765184 to 7317126146011922432. Current Snapshot: MvccSnapshot[applied={T|T < 7317126146284765184}]
I20260811 00:22:30.619575 8162 mvcc.cc:205] Tried to move back new op lower bound from 7317126146284765184 to 7317126146011922432. Current Snapshot: MvccSnapshot[applied={T|T < 7317126146284765184}]
I20260811 00:22:30.634889 7174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:22:30.635233 7174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:22:30.640038 7174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d358568f8cd47b1832ad035aea71b0b: Sending DeleteTablet for 3 replicas of tablet a08c33c9bd22489db284b79d6c954c35
I20260811 00:22:30.641433 7960 tablet_service.cc:1514] Processing DeleteTablet for tablet a08c33c9bd22489db284b79d6c954c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:30 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:59030
I20260811 00:22:30.641462 7704 tablet_service.cc:1514] Processing DeleteTablet for tablet a08c33c9bd22489db284b79d6c954c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:30 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:60550
I20260811 00:22:30.647976 7433 tablet_service.cc:1514] Processing DeleteTablet for tablet a08c33c9bd22489db284b79d6c954c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:30 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:44910
I20260811 00:22:30.651953 8196 tablet_replica.cc:330] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: stopping tablet replica
I20260811 00:22:30.652945 8200 tablet_replica.cc:330] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: stopping tablet replica
I20260811 00:22:30.653751 8196 raft_consensus.cc:2229] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:22:30.654400 8196 raft_consensus.cc:2258] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:30.655021 8200 raft_consensus.cc:2229] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:22:30.655306 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 7247
I20260811 00:22:30.655908 8200 raft_consensus.cc:2258] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:30.660967 8195 tablet_replica.cc:330] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: stopping tablet replica
I20260811 00:22:30.661453 8195 raft_consensus.cc:2229] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:22:30.662034 8195 raft_consensus.cc:2258] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:30.665735 8200 ts_tablet_manager.cc:1905] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:30.668391 8196 ts_tablet_manager.cc:1905] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:30.670882 8195 ts_tablet_manager.cc:1905] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:30.675536 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.656262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.16.71.129:37467)
0811 00:22:30.656557 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.656563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.656587 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:22:30.656817 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.656823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.656968 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.657090 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:30.659066 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.660305 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.662600 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.663330 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.663461 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.667271 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:30.667288 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:30.667309 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:30.667374 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:30.671763 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.672870 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.672879 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.672883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.673096 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.673584 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.673591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.673594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.674118 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:30.674467 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.674826 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.675129 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260811 00:22:30.676290 7550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:30.678357 8195 ts_tablet_manager.cc:1918] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:30.678581 8195 log.cc:1196] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/a08c33c9bd22489db284b79d6c954c35
I20260811 00:22:30.679179 8195 ts_tablet_manager.cc:1939] T a08c33c9bd22489db284b79d6c954c35 P cf15eaef4d704df8b99e271b9bd6122e: Deleting consensus metadata
I20260811 00:22:30.681947 7145 catalog_manager.cc:4841] TS cf15eaef4d704df8b99e271b9bd6122e (127.16.71.130:35883): tablet a08c33c9bd22489db284b79d6c954c35 (table test-table [id=d9d538ca836f4de481d732968528b3f1]) successfully deleted
I20260811 00:22:30.685737 8196 ts_tablet_manager.cc:1918] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:30.685966 8196 log.cc:1196] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/a08c33c9bd22489db284b79d6c954c35
I20260811 00:22:30.686513 8196 ts_tablet_manager.cc:1939] T a08c33c9bd22489db284b79d6c954c35 P be87ac007c8a4145bbb3a0ec929444bf: Deleting consensus metadata
I20260811 00:22:30.688827 7146 catalog_manager.cc:4841] TS be87ac007c8a4145bbb3a0ec929444bf (127.16.71.131:46583): tablet a08c33c9bd22489db284b79d6c954c35 (table test-table [id=d9d538ca836f4de481d732968528b3f1]) successfully deleted
I20260811 00:22:30.786979 8200 ts_tablet_manager.cc:1918] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:30.787313 8200 log.cc:1196] T a08c33c9bd22489db284b79d6c954c35 P ad629f3b7117495580576d9b13c84789: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/a08c33c9bd22489db284b79d6c954c35
I20260811 00:22:30.788825 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 7247
W20260811 00:22:30.817464 7148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.71.129:37467 (error 108)
I20260811 00:22:30.817690 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 7563
I20260811 00:22:30.819908 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.819385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57432)
0811 00:22:30.819641 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.819743 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260811 00:22:30.820187 7148 catalog_manager.cc:4584] TS ad629f3b7117495580576d9b13c84789 (127.16.71.129:37467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a08c33c9bd22489db284b79d6c954c35: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
I20260811 00:22:30.835003 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.818721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.16.71.130:35883)
0811 00:22:30.818907 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.818913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.818934 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:22:30.819003 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.819007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.819092 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.819214 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:30.821095 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.821858 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.823685 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.824088 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.824190 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.827657 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:30.827678 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:30.827697 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:30.827747 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:30.831168 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.832390 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.832404 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.832408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.832662 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.833170 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.833175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.833177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.833667 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:30.833975 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.834416 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.834633 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:22:30.835733 7812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:30.874145 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.873555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57444)
0811 00:22:30.873828 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.873962 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 00:22:30.875512 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.875004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57448)
0811 00:22:30.875191 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.875311 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 00:22:30.934521 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 7563
I20260811 00:22:30.944990 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.944268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57458)
0811 00:22:30.944607 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.944779 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 00:22:30.946789 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.946145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57462)
0811 00:22:30.946429 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:30.946557 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 00:22:30.965287 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 7824
I20260811 00:22:30.981011 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:30.966228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.16.71.131:46583)
0811 00:22:30.966704 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:30.966709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:30.966727 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:22:30.966786 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:30.966790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:30.966870 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:30.966968 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:30.968477 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.969299 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.970830 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:30.971256 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:30.971369 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:30.974465 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:30.974483 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:30.974505 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:30.974572 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:30.977671 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.978643 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.978653 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.978656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.978844 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:30.979236 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:30.979239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:30.979241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:30.979638 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:30.979887 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:30.980293 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 00:22:30.980504 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":239,"threads_started":1}
I20260811 00:22:30.981751 8086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:31.027794 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:31.027260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57468)
0811 00:22:31.027531 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:31.027628 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 00:22:31.029001 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:31.028514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57476)
0811 00:22:31.028767 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:31.028852 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260811 00:22:31.074198 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 7824
I20260811 00:22:31.103647 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 7112
I20260811 00:22:31.105067 7234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:31.223717 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:31.223132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:37467 (local address 127.0.0.1:57482)
0811 00:22:31.223331 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:31.223496 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 00:22:31.223680 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 7112
I20260811 00:22:31.263461 16670 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 00:22:31.610821 16670 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 00:22:31.611097 16670 mini_ranger.cc:66] Stopping Ranger...
I20260811 00:22:31.958213 16670 mini_ranger.cc:68] Stopped Ranger
2026-08-11 00:22:31.958 UTC [5345] LOG: received smart shutdown request
2026-08-11 00:22:31.962 UTC [5345] LOG: background worker "logical replication launcher" (PID 5364) exited with exit code 1
2026-08-11 00:22:31.962 UTC [5352] LOG: shutting down
2026-08-11 00:22:32.174 UTC [5345] LOG: database system is shut down
2026-08-11T00:22:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (58634 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:22:32 dist-test-slave-f7th krb5kdc[8230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:22:32 dist-test-slave-f7th krb5kdc[8230](info): set up 2 sockets
Aug 11 00:22:32 dist-test-slave-f7th krb5kdc[8230](info): commencing operation
krb5kdc: starting...
W20260811 00:22:33.615808 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:22:33 dist-test-slave-f7th krb5kdc[8230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:22:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:22:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:22:33.917385 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:35757
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:35757,127.16.71.189:44593,127.16.71.188:39471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:34.315251 8255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:34.315639 8255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:34.315778 8255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:34.323622 8255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:34.323786 8255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:34.323861 8255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:34.323940 8255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:34.323999 8255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:34.335000 8255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:35757,127.16.71.189:44593,127.16.71.188:39471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:35757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:34.336606 8255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:34.338243 8255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:34.351848 8264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:34.351815 8271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:34.351761 8268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:34.352887 8255 server_base.cc:1034] running on GCE node
I20260811 00:22:34.353830 8255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:34.355607 8255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:34.356925 8255 hybrid_clock.cc:648] HybridClock initialized: now 1786407754356839 us; error 67 us; skew 500 ppm
I20260811 00:22:34.357400 8255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:34 dist-test-slave-f7th krb5kdc[8230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:34.361048 8255 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:22:34.362850 8255 webserver.cc:460] Webserver started at http://127.16.71.190:39149/ using document root <none> and password file <none>
I20260811 00:22:34.363672 8255 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:34.363824 8255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:34.364347 8255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:34.368017 8255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "3315839146724e4b9fc2cac80ac54a23"
format_stamp: "Formatted at 2026-08-11 00:22:34 on dist-test-slave-f7th"
server_key: "e4d7b5925f10eeda9727727860b048de"
server_key_iv: "5a38dc46876983df8a40bf17e8ae303d"
server_key_version: "encryptionkey@0"
I20260811 00:22:34.369143 8255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "3315839146724e4b9fc2cac80ac54a23"
format_stamp: "Formatted at 2026-08-11 00:22:34 on dist-test-slave-f7th"
server_key: "e4d7b5925f10eeda9727727860b048de"
server_key_iv: "5a38dc46876983df8a40bf17e8ae303d"
server_key_version: "encryptionkey@0"
I20260811 00:22:34.375542 8255 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:22:34.380229 8287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:34.382009 8255 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 00:22:34.382252 8255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "3315839146724e4b9fc2cac80ac54a23"
format_stamp: "Formatted at 2026-08-11 00:22:34 on dist-test-slave-f7th"
server_key: "e4d7b5925f10eeda9727727860b048de"
server_key_iv: "5a38dc46876983df8a40bf17e8ae303d"
server_key_version: "encryptionkey@0"
I20260811 00:22:34.382491 8255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:34.416584 8255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:34.421588 8255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:34.421926 8255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:34.442873 8255 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:35757
I20260811 00:22:34.442911 8403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:35757 every 8 connection(s)
I20260811 00:22:34.444689 8255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:22:34.449605 8404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:34.450524 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 8255
I20260811 00:22:34.450750 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:22:34.451545 16670 external_mini_cluster.cc:1428] Setting key cefd9fb8753ac4f0bd0d58524a9a62f4
I20260811 00:22:34.457828 8404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:22:34 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.189@KRBTEST.COM" created.
I20260811 00:22:34.485045 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.462036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:35757 (local address 127.0.0.1:33466)
0811 00:22:34.463050 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.463125 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:34.465660 (+ 2535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:34.466736 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:34.466743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:34.467606 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:34.468000 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:34.468023 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:34.469751 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:34.469756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:34.470934 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:34.470950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:34.472149 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:34.472157 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:34.472441 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:34.473195 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:34.473223 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:34.476323 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:34.480306 (+ 3983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:34.480321 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:34.480337 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:34.480891 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:34.481479 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:34.481484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:34.481488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:34.481675 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:34.482648 (+ 973us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:34.482656 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:34.482932 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 00:22:34.483213 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 00:22:34.483893 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":190,"thread_start_us":415,"threads_started":1}
I20260811 00:22:34.485092 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.463554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.16.71.190:35757)
0811 00:22:34.464256 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:34.464263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:34.464333 (+ 70us) server_negotiation.cc:407] Connection header received
0811 00:22:34.466135 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:34.466142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:34.466452 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:34.466674 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:34.468227 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:34.469583 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:34.471162 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:34.471859 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:34.472077 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:34.476663 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:34.476677 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:34.476699 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:34.476739 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:34.480119 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:34.481087 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:34.481095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:34.481097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:34.481356 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:34.481799 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:34.481805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:34.481809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:34.482419 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:34.482683 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:34.483385 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 00:22:34.483717 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":512,"thread_start_us":342,"threads_started":1}
I20260811 00:22:34.489218 8404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:22:34.490865 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.490335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38388)
0811 00:22:34.490572 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.490715 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 00:22:34.491230 8293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
I20260811 00:22:34.493172 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.492728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38398)
0811 00:22:34.492919 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.493010 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 00:22:34.493402 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:44593: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab.
Entry for principal kudu/127.16.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab.
I20260811 00:22:34.525667 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:44593
--webserver_interface=127.16.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab
--principal=kudu/127.16.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:35757,127.16.71.189:44593,127.16.71.188:39471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:22:34.542910 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } attempt: 1
I20260811 00:22:34.544368 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.543729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38410)
0811 00:22:34.543995 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.544192 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 00:22:34.545691 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.545247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38426)
0811 00:22:34.545422 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.545520 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:22:34.545979 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:44593: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
I20260811 00:22:34.614290 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } attempt: 2
I20260811 00:22:34.615892 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.615116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38442)
0811 00:22:34.615515 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.615630 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260811 00:22:34.617424 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.616931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38458)
0811 00:22:34.617135 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.617245 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 00:22:34.617687 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:44593: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
I20260811 00:22:34.709003 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } attempt: 3
I20260811 00:22:34.710175 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.709711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38470)
0811 00:22:34.709927 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.710035 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 00:22:34.711467 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.710985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38476)
0811 00:22:34.711184 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.711294 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 00:22:34.711689 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:44593: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
I20260811 00:22:34.854979 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } attempt: 4
I20260811 00:22:34.856294 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.855666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38484)
0811 00:22:34.856048 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.856145 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 00:22:34.857748 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:34.857315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38488)
0811 00:22:34.857484 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:34.857596 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 00:22:34.857991 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:44593: Network error: Client connection negotiation failed: client connection to 127.16.71.189:44593: connect: Connection refused (error 111)
W20260811 00:22:34.897601 8414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:34.897960 8414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:34.898110 8414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:34.906165 8414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:34.906307 8414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:34.906368 8414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:34.906455 8414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:34.906522 8414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:34.917230 8414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:35757,127.16.71.189:44593,127.16.71.188:39471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab
--principal=kudu/127.16.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:34.918711 8414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:34.920105 8414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:34.933089 8421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:34.933132 8426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:34.933154 8422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:34.934403 8414 server_base.cc:1034] running on GCE node
I20260811 00:22:34.935345 8414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:34.937044 8414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:34.938293 8414 hybrid_clock.cc:648] HybridClock initialized: now 1786407754938229 us; error 52 us; skew 500 ppm
I20260811 00:22:34.938735 8414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:34 dist-test-slave-f7th krb5kdc[8230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:34.942263 8414 init.cc:377] Logged in from keytab as kudu/127.16.71.189@KRBTEST.COM (short username kudu)
I20260811 00:22:34.944335 8414 webserver.cc:460] Webserver started at http://127.16.71.189:43317/ using document root <none> and password file <none>
I20260811 00:22:34.945322 8414 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:34.945461 8414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:34.945847 8414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:34.949621 8414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data/instance:
uuid: "a51575ac2579457d83f617edc7495a5e"
format_stamp: "Formatted at 2026-08-11 00:22:34 on dist-test-slave-f7th"
server_key: "1b30c97f908c5bc1e81163ac5c2c20fc"
server_key_iv: "f57c2b8c53f2bba65f9ce644905058e4"
server_key_version: "encryptionkey@0"
I20260811 00:22:34.950655 8414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/wal/instance:
uuid: "a51575ac2579457d83f617edc7495a5e"
format_stamp: "Formatted at 2026-08-11 00:22:34 on dist-test-slave-f7th"
server_key: "1b30c97f908c5bc1e81163ac5c2c20fc"
server_key_iv: "f57c2b8c53f2bba65f9ce644905058e4"
server_key_version: "encryptionkey@0"
I20260811 00:22:34.957157 8414 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:22:34.961488 8438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:34.963178 8414 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:22:34.963387 8414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/wal
uuid: "a51575ac2579457d83f617edc7495a5e"
format_stamp: "Formatted at 2026-08-11 00:22:34 on dist-test-slave-f7th"
server_key: "1b30c97f908c5bc1e81163ac5c2c20fc"
server_key_iv: "f57c2b8c53f2bba65f9ce644905058e4"
server_key_version: "encryptionkey@0"
I20260811 00:22:34.963616 8414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:34.989956 8414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:34.994992 8414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:34.995282 8414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:35.014631 8414 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.189:44593
I20260811 00:22:35.014665 8542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.189:44593 every 8 connection(s)
I20260811 00:22:35.016263 8414 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
I20260811 00:22:35.018325 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 8414
I20260811 00:22:35.018524 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-1/wal/instance
I20260811 00:22:35.019011 16670 external_mini_cluster.cc:1428] Setting key 311ae355baa671ebc23b498676060ad6
I20260811 00:22:35.020792 8543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:35.028252 8543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.188@KRBTEST.COM" created.
I20260811 00:22:35.052820 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.031370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.16.71.190:35757)
0811 00:22:35.032019 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.032027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.033219 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 00:22:35.035366 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.035373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.035482 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.035633 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.037568 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.038688 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.040454 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.041180 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.041335 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.045732 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.045739 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.045743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.045786 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.048987 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.050112 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.050118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.050121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.050294 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.050764 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.050769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.050773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.051204 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.051368 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.052132 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.052385 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":347,"threads_started":1}
I20260811 00:22:35.053741 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.032058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:35757 (local address 127.0.0.1:33482)
0811 00:22:35.033013 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.033066 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.035009 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.035882 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.035892 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.036526 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.037291 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.037314 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.038868 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.038872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.040248 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.040265 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.041491 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.041498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.041740 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.042553 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.042586 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.045385 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.049218 (+ 3833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.049230 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.049254 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.049902 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.050417 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.050420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.050423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.050632 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.051329 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.051335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.051723 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.051983 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.052426 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":375,"threads_started":1}
I20260811 00:22:35.055855 8543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:22:35.073937 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.056791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38494)
0811 00:22:35.057021 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.057042 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.057178 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.058374 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.058378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.058791 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.058941 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.058950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.060848 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.060853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.062134 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.062154 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.063093 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.063103 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.063317 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.063971 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.063995 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.066720 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.071284 (+ 4564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.071289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.071292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.071741 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.072225 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.072229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.072231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.072372 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.073162 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.073166 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.073322 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.073436 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.073662 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 00:22:35.074290 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.057042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.16.71.189:44593)
0811 00:22:35.057819 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.057827 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.057859 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:22:35.057920 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.057928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.058193 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.058354 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.059099 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.060670 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.062314 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.062915 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.063031 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.067026 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.067049 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.067073 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.067117 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.071067 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.071911 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.071921 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.071924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.072104 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.072504 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.072509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.072512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.073035 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.073267 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.073612 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.073887 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":417,"threads_started":1}
I20260811 00:22:35.077003 8543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:22:35.078241 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.077643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43900)
0811 00:22:35.077876 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.078026 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 00:22:35.078516 8442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.080291 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.079836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43916)
0811 00:22:35.080020 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.080116 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 00:22:35.080538 8543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab.
Entry for principal kudu/127.16.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab.
I20260811 00:22:35.088896 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:39471
--webserver_interface=127.16.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab
--principal=kudu/127.16.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:35757,127.16.71.189:44593,127.16.71.188:39471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:22:35.129933 8543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 1
I20260811 00:22:35.131256 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.130654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43922)
0811 00:22:35.130902 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.131043 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:22:35.132417 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.131982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43938)
0811 00:22:35.132154 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.132268 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 00:22:35.132730 8543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.158033 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } attempt: 5
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:22:35.174372 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.158898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38500)
0811 00:22:35.159144 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.159168 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.159319 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.160070 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.160076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.160543 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.160891 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.160905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.162384 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.162389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.163293 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.163307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.164500 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.164505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.164729 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.165429 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.165450 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.167663 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.171762 (+ 4099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.171768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.171771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.172251 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.172740 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.172744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.172747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.172904 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.173545 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.173549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.173701 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.173820 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.174110 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 00:22:35.174538 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.158964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.16.71.189:44593)
0811 00:22:35.159647 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.159653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.159673 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:22:35.159738 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.159744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.159853 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.159966 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.161095 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.162204 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.163498 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.164302 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.164446 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.167937 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.167944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.167949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.167997 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.171526 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.172405 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.172411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.172415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.172597 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.173018 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.173023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.173027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.173429 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.173602 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.173956 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.174217 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472}
I20260811 00:22:35.175745 8404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:22:35.177023 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.176524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43944)
0811 00:22:35.176750 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.176837 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 00:22:35.178047 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.177689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43954)
0811 00:22:35.177811 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.177899 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 00:22:35.178273 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.201030 8543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 2
I20260811 00:22:35.202457 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.201852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43962)
0811 00:22:35.202158 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.202269 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 00:22:35.203683 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.203272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43970)
0811 00:22:35.203435 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.203534 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 00:22:35.203938 8543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.229547 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 1
I20260811 00:22:35.230779 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.230264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43972)
0811 00:22:35.230501 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.230605 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:22:35.232133 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.231740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43974)
0811 00:22:35.231912 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.232000 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 00:22:35.232410 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.295245 8543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 3
I20260811 00:22:35.296444 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.295937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43982)
0811 00:22:35.296177 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.296272 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:22:35.297829 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.297259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:43996)
0811 00:22:35.297594 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.297689 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
W20260811 00:22:35.298128 8543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.300664 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 2
I20260811 00:22:35.301879 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.301365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44000)
0811 00:22:35.301615 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.301707 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 00:22:35.302996 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.302603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44006)
0811 00:22:35.302759 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.302859 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 00:22:35.303213 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.409471 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 3
I20260811 00:22:35.410781 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.410237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44012)
0811 00:22:35.410487 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.410596 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 00:22:35.411964 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.411548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44016)
0811 00:22:35.411752 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.411841 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 00:22:35.412227 8404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
I20260811 00:22:35.441399 8543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 4
I20260811 00:22:35.442970 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.442261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44032)
0811 00:22:35.442578 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.442701 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 00:22:35.444646 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.444058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44044)
0811 00:22:35.444284 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.444392 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 00:22:35.444953 8543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:39471: Network error: Client connection negotiation failed: client connection to 127.16.71.188:39471: connect: Connection refused (error 111)
W20260811 00:22:35.470589 8565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:35.470911 8565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:35.471047 8565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:35.478861 8565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:35.479014 8565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:35.479087 8565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:35.479177 8565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:35.479246 8565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:35.490094 8565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:35757,127.16.71.189:44593,127.16.71.188:39471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab
--principal=kudu/127.16.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:39471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:35.491621 8565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:35.493162 8565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:35.506462 8573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:35.506652 8574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:35.506677 8578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:35.508507 8565 server_base.cc:1034] running on GCE node
I20260811 00:22:35.509614 8565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:35.511579 8565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:35.512874 8565 hybrid_clock.cc:648] HybridClock initialized: now 1786407755512809 us; error 58 us; skew 500 ppm
I20260811 00:22:35.513337 8565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407755, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:35.517405 8565 init.cc:377] Logged in from keytab as kudu/127.16.71.188@KRBTEST.COM (short username kudu)
I20260811 00:22:35.519466 8565 webserver.cc:460] Webserver started at http://127.16.71.188:41845/ using document root <none> and password file <none>
I20260811 00:22:35.520308 8565 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:35.520454 8565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:35.520830 8565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:35.524386 8565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data/instance:
uuid: "a89a894deaa14deb8512e35c5c20beca"
format_stamp: "Formatted at 2026-08-11 00:22:35 on dist-test-slave-f7th"
server_key: "fefb2e5da6b0d04090a1f5f411ae5378"
server_key_iv: "59819effcc97d48b18d68b4755e888ec"
server_key_version: "encryptionkey@0"
I20260811 00:22:35.525439 8565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/wal/instance:
uuid: "a89a894deaa14deb8512e35c5c20beca"
format_stamp: "Formatted at 2026-08-11 00:22:35 on dist-test-slave-f7th"
server_key: "fefb2e5da6b0d04090a1f5f411ae5378"
server_key_iv: "59819effcc97d48b18d68b4755e888ec"
server_key_version: "encryptionkey@0"
I20260811 00:22:35.532325 8565 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:22:35.536825 8591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:35.538833 8565 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:22:35.539060 8565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/wal
uuid: "a89a894deaa14deb8512e35c5c20beca"
format_stamp: "Formatted at 2026-08-11 00:22:35 on dist-test-slave-f7th"
server_key: "fefb2e5da6b0d04090a1f5f411ae5378"
server_key_iv: "59819effcc97d48b18d68b4755e888ec"
server_key_version: "encryptionkey@0"
I20260811 00:22:35.539309 8565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:35.552505 8565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:35.557677 8565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:35.558055 8565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:35.577593 8565 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.188:39471
I20260811 00:22:35.577641 8690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.188:39471 every 8 connection(s)
I20260811 00:22:35.579176 8565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
I20260811 00:22:35.583410 8691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:35.589401 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 8565
I20260811 00:22:35.589468 8404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 4
I20260811 00:22:35.589660 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407527335556-16670-0/minicluster-data/master-2/wal/instance
I20260811 00:22:35.590260 16670 external_mini_cluster.cc:1428] Setting key d4d104778c9afa6aba8bdfde3b847952
I20260811 00:22:35.591771 8691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407755, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
I20260811 00:22:35.615089 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.595776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33490 (local address 127.16.71.190:35757)
0811 00:22:35.596334 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.596340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.597110 (+ 770us) server_negotiation.cc:407] Connection header received
0811 00:22:35.597382 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.597387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.597462 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.597542 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.599800 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.600844 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.602489 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.602947 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.603056 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.608167 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.608175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.608180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.608237 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.611208 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.612348 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.612354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.612358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.612508 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.613107 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.613113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.613117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.613473 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.613636 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.614372 (+ 736us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.614657 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":426,"thread_start_us":273,"threads_started":1}
I20260811 00:22:35.615442 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.590218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44048)
0811 00:22:35.590538 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.590560 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.590698 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.596402 (+ 5704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.596408 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.597023 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.597258 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.597273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.600470 (+ 3197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.600478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.602133 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.602158 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.603162 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.603168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.603369 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.604127 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.604148 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.606580 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.611547 (+ 4967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.611553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.611556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.612108 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.612721 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.612726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.612730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.612901 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.614006 (+ 1105us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.614012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.614203 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.614355 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.615103 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":411}
I20260811 00:22:35.616554 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.595858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:35757 (local address 127.0.0.1:33490)
0811 00:22:35.596947 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.597020 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.597235 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.597807 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.597814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.598826 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.599530 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.599560 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.601038 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.601042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.602325 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.602341 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.603216 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.603224 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.603388 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.604807 (+ 1419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.604838 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.607896 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.611431 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.611443 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.611459 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.612221 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.612671 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.612676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.612680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.612948 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.613629 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.613638 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.613919 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.614239 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.614844 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":6,"thread_start_us":671,"threads_started":1}
I20260811 00:22:35.617268 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.592696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.16.71.188:39471)
0811 00:22:35.593388 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.593393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.593414 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:22:35.595413 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.595422 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.596264 (+ 842us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.597471 (+ 1207us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.598555 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.600339 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.603292 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.603881 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.604005 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.609204 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.609212 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.609217 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.609260 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.612826 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.613800 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.613805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.613808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.613959 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.614525 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.614530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.614534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.614904 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.615232 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.616525 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.616832 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":561,"spinlock_wait_cycles":27776,"thread_start_us":385,"threads_started":1}
I20260811 00:22:35.617832 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.591334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44048 (local address 127.16.71.188:39471)
0811 00:22:35.592678 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.592696 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.592757 (+ 61us) server_negotiation.cc:407] Connection header received
0811 00:22:35.595386 (+ 2629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.595414 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.595917 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.597482 (+ 1565us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.598139 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.600273 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.602343 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.602969 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.603202 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.606911 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.606926 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.606947 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.606993 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.611356 (+ 4363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.612314 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.612325 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.612329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.612568 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.613013 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.613018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.613022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.613855 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.614036 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.614654 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.617229 (+ 2575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2508,"server-negotiator.queue_time_us":1021,"spinlock_wait_cycles":14208,"thread_start_us":826,"threads_started":1}
I20260811 00:22:35.618605 8691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:22:35.626251 8404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23: Bootstrap starting.
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407755, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:22:35.631863 8404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:35.633735 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.619460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:44593 (local address 127.0.0.1:38502)
0811 00:22:35.619780 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.619799 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.619927 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.620303 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.620344 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.620772 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.620939 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.620948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.622541 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.622547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.623482 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.623498 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.624279 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.624284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.624488 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.625133 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.625152 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.627432 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.630730 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.630736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.630740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.631354 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.631822 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.631828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.631832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.632022 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.632673 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.632678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.632854 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.632990 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.633332 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 00:22:35.633657 8404 log.cc:824] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:35.633955 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.619438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.16.71.189:44593)
0811 00:22:35.619643 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.619649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.619876 (+ 227us) server_negotiation.cc:407] Connection header received
0811 00:22:35.620058 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.620063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.620150 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.620262 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.621113 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.622319 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.623663 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.624114 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.624237 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.627684 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.627690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.627694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.627735 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.630539 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.631513 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.631519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.631521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.631687 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.632179 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.632184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.632186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.632505 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.632722 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.633129 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.633580 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:35.635280 8691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:22:35.639039 8404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23: No bootstrap required, opened a new log
I20260811 00:22:35.643682 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.622661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33492 (local address 127.16.71.190:35757)
0811 00:22:35.622884 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.622894 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.622913 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:22:35.622979 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.622984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.623066 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.623167 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.625567 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.626645 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.629154 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.629844 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.629970 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.636181 (+ 6211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.636188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.636193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.636236 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.640120 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.641202 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.641208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.641210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.641375 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.641812 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.641817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.641819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.642174 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.642495 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.642862 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.643123 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407755, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
I20260811 00:22:35.650791 8404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.651461 8404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:35.651618 8404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3315839146724e4b9fc2cac80ac54a23, State: Initialized, Role: FOLLOWER
I20260811 00:22:35.652412 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.636270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44062)
0811 00:22:35.636823 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.636840 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.636974 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.637269 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.637274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.637725 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.637896 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.637905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.638931 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.638937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.639859 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.639874 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.640765 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.640771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.640990 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.641639 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.641663 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.645020 (+ 3357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.649215 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.649221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.649225 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.649763 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.650263 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.650269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.650273 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.650481 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.651282 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.651289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.651469 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.651658 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.652033 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437}
I20260811 00:22:35.652356 8404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.653334 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.636288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.16.71.188:39471)
0811 00:22:35.636452 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.636457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.636927 (+ 470us) server_negotiation.cc:407] Connection header received
0811 00:22:35.637085 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.637090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.637152 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.637230 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.638060 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.638775 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.640005 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.640534 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.640698 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.645261 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.645268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.645272 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.645316 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.649067 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.649915 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.649921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.649924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.650124 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.650616 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.650621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.650624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.651136 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.651313 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.652781 (+ 1468us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.653028 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 00:22:35.655514 8404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:22:35.660076 8691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: Bootstrap starting.
I20260811 00:22:35.663697 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.645789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38516 (local address 127.16.71.189:44593)
0811 00:22:35.645986 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.645992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.646016 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:22:35.646080 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.646086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.646164 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.646270 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.648099 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.649339 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.651057 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.651708 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.651831 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.657872 (+ 6041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.657879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.657883 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.657929 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.660781 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.661646 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.661652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.661654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.661816 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.662206 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.662210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.662213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.662524 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.662803 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.663093 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.663336 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:22:35.654101 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:35.665112 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:22:35.665922 8691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:35.667809 8691 log.cc:824] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:35.669106 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:22:35.672394 8691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: No bootstrap required, opened a new log
W20260811 00:22:35.675439 8300 tablet.cc:2369] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:22:35.685887 8691 raft_consensus.cc:348] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.686702 8691 raft_consensus.cc:374] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:35.686937 8691 raft_consensus.cc:729] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89a894deaa14deb8512e35c5c20beca, State: Initialized, Role: FOLLOWER
I20260811 00:22:35.688074 8691 consensus_queue.cc:260] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.693352 8724 sys_catalog.cc:455] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:35.696420 8724 sys_catalog.cc:458] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:22:35.696208 8691 sys_catalog.cc:564] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:22:35.711045 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.683627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33500 (local address 127.16.71.190:35757)
0811 00:22:35.684120 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.684129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.690311 (+ 6182us) server_negotiation.cc:407] Connection header received
0811 00:22:35.690647 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.690655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.690754 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.690839 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.693785 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.695036 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.697448 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.698166 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.698272 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.702818 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.702827 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.702832 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.702889 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.706965 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.707994 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.708014 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.708018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.708219 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.708901 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.708907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.708911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.709353 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.709675 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.710319 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.710611 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260811 00:22:35.714713 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.684364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.16.71.189:44593)
0811 00:22:35.684546 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.684551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.689661 (+ 5110us) server_negotiation.cc:407] Connection header received
0811 00:22:35.690548 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.690556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.690668 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.690763 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.697265 (+ 6502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.698373 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.702190 (+ 3817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.702886 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.703000 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.706367 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.706375 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.706380 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.706443 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.710134 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.711069 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.711075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.711079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.711264 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.711756 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.711762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.711764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.712189 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.713833 (+ 1644us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.714060 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.714348 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 00:22:35.727506 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.690439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.16.71.188:39471)
0811 00:22:35.690661 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.690667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.698028 (+ 7361us) server_negotiation.cc:407] Connection header received
0811 00:22:35.699158 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.699166 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.699267 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.699347 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.705313 (+ 5966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.706551 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.711320 (+ 4769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.712164 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.712294 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.717661 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.717670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.717674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.717724 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.721622 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.722776 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.722784 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.722788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.722979 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.724362 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.724369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.724373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.724875 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.725216 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.726753 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.727077 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 00:22:35.731061 8738 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:22:35.731222 8738 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:22:35.723991 8255 master_runner.cc:186] Error getting master registration for 127.16.71.189:44593: OK, instance_id { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" instance_seqno: 1786407755009107 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:22:35.737509 8255 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:22:35.744233 8543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } attempt: 5
Aug 11 00:22:35 dist-test-slave-f7th krb5kdc[8230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407754, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
W20260811 00:22:35.759951 8752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:22:35.760201 8752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:22:35.749126 8565 master_runner.cc:186] Error getting master registration for 127.16.71.189:44593: OK, instance_id { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" instance_seqno: 1786407755009107 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:22:35.762253 8565 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:22:35.762998 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.744924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:39471 (local address 127.0.0.1:44084)
0811 00:22:35.745108 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:35.745129 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:35.745271 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:35.745639 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:35.745644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:35.746214 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:35.746432 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.746445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.748194 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.748200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.749806 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:35.749827 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.751110 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.751119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:35.751391 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.752154 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:35.752182 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:35.755129 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:35.760135 (+ 5006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.760141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.760145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.760741 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.761160 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:35.761164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:35.761167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:35.761319 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:35.762039 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:35.762044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:35.762233 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 00:22:35.762370 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 00:22:35.762641 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 00:22:35.764037 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:35.745167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.16.71.188:39471)
0811 00:22:35.745330 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:35.745336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:35.745353 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:22:35.745406 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:35.745411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:35.745495 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:35.745576 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:35.746616 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.748006 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.750069 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:35.750892 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:35.751823 (+ 931us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:35.755363 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:35.755369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:35.755373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:35.755419 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:35.759883 (+ 4464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.760870 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.760880 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.760884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.761030 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:35.761462 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:35.761467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:35.761469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:35.761891 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:35.762055 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:35.763380 (+ 1325us) server_negotiation.cc:299] Negotiation successful
0811 00:22:35.763664 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 00:22:35.770776 8543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: Bootstrap starting.
I20260811 00:22:35.777729 8543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:35.778972 8543 log.cc:824] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:35.782219 8543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: No bootstrap required, opened a new log
I20260811 00:22:35.792858 8543 raft_consensus.cc:348] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.793294 8543 raft_consensus.cc:374] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:35.793447 8543 raft_consensus.cc:729] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51575ac2579457d83f617edc7495a5e, State: Initialized, Role: FOLLOWER
I20260811 00:22:35.794130 8543 consensus_queue.cc:260] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.796103 8763 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:35.798679 8763 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:22:35.797859 8543 sys_catalog.cc:564] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:22:35.810595 8603 tablet.cc:2369] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:22:35.818076 8414 master_runner.cc:263] No leader master found from master a51575ac2579457d83f617edc7495a5e
I20260811 00:22:35.818243 8414 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 00:22:35.831002 8776 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:22:35.831207 8776 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:22:35.871600 8703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:22:35.872016 8703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.873952 8703 leader_election.cc:288] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a51575ac2579457d83f617edc7495a5e (127.16.71.189:44593), a89a894deaa14deb8512e35c5c20beca (127.16.71.188:39471)
I20260811 00:22:35.875797 8485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3315839146724e4b9fc2cac80ac54a23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a51575ac2579457d83f617edc7495a5e" is_pre_election: true
I20260811 00:22:35.877117 8640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3315839146724e4b9fc2cac80ac54a23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89a894deaa14deb8512e35c5c20beca" is_pre_election: true
I20260811 00:22:35.877823 8640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3315839146724e4b9fc2cac80ac54a23 in term 0.
I20260811 00:22:35.878719 8293 leader_election.cc:302] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3315839146724e4b9fc2cac80ac54a23, a89a894deaa14deb8512e35c5c20beca; no voters:
I20260811 00:22:35.879446 8485 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3315839146724e4b9fc2cac80ac54a23 in term 0.
I20260811 00:22:35.880725 8703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:22:35.880930 8703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:22:35.881044 8703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:35.886940 8703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.890702 8485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3315839146724e4b9fc2cac80ac54a23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a51575ac2579457d83f617edc7495a5e"
I20260811 00:22:35.891003 8485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:35.895186 8485 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3315839146724e4b9fc2cac80ac54a23 in term 1.
I20260811 00:22:35.895934 8293 leader_election.cc:302] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3315839146724e4b9fc2cac80ac54a23, a51575ac2579457d83f617edc7495a5e; no voters:
I20260811 00:22:35.899346 8703 leader_election.cc:288] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [CANDIDATE]: Term 1 election: Requested vote from peers a51575ac2579457d83f617edc7495a5e (127.16.71.189:44593), a89a894deaa14deb8512e35c5c20beca (127.16.71.188:39471)
I20260811 00:22:35.899632 8703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:22:35.899794 8703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [term 1 LEADER]: Becoming Leader. State: Replica: 3315839146724e4b9fc2cac80ac54a23, State: Running, Role: LEADER
I20260811 00:22:35.900452 8703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } }
I20260811 00:22:35.916316 8640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3315839146724e4b9fc2cac80ac54a23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89a894deaa14deb8512e35c5c20beca"
I20260811 00:22:35.916796 8640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:35.920576 8640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3315839146724e4b9fc2cac80ac54a23 in term 1.
I20260811 00:22:35.923359 8778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3315839146724e4b9fc2cac80ac54a23. Latest consensus state: current_term: 1 leader_uuid: "3315839146724e4b9fc2cac80ac54a23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:35.923899 8778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:35.929230 8781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:22:35.932021 8781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:22:35.956444 8640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [term 1 FOLLOWER]: Refusing update from remote peer 3315839146724e4b9fc2cac80ac54a23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:35.957792 8778 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:22:35.965992 8485 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [term 1 FOLLOWER]: Refusing update from remote peer 3315839146724e4b9fc2cac80ac54a23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:35.971056 8778 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:22:35.998450 8449 tablet.cc:2369] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:22:36.008253 8763 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3315839146724e4b9fc2cac80ac54a23. Latest consensus state: current_term: 1 leader_uuid: "3315839146724e4b9fc2cac80ac54a23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:36.008699 8763 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:22:36.012516 8782 mvcc.cc:205] Tried to move back new op lower bound from 7317126168381775872 to 7317126168204636160. Current Snapshot: MvccSnapshot[applied={T|T < 7317126168381775872}]
I20260811 00:22:36.027478 8785 mvcc.cc:205] Tried to move back new op lower bound from 7317126168381775872 to 7317126168204636160. Current Snapshot: MvccSnapshot[applied={T|T < 7317126168381775872}]
I20260811 00:22:36.037397 8724 sys_catalog.cc:455] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3315839146724e4b9fc2cac80ac54a23. Latest consensus state: current_term: 1 leader_uuid: "3315839146724e4b9fc2cac80ac54a23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:36.037788 8724 sys_catalog.cc:458] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:22:36.043233 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3315839146724e4b9fc2cac80ac54a23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:36.043675 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:36.051043 8778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3315839146724e4b9fc2cac80ac54a23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:36.051396 8778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:36.052142 8783 mvcc.cc:205] Tried to move back new op lower bound from 7317126168381775872 to 7317126168204636160. Current Snapshot: MvccSnapshot[applied={T|T < 7317126168381775872}]
I20260811 00:22:36.065434 8763 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3315839146724e4b9fc2cac80ac54a23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:36.065793 8763 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:22:36.070611 8781 catalog_manager.cc:1347] Generated new cluster ID: 25597673d8ec47698e24260fbde48c68
I20260811 00:22:36.070760 8781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:22:36.088855 8724 sys_catalog.cc:455] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3315839146724e4b9fc2cac80ac54a23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3315839146724e4b9fc2cac80ac54a23" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 35757 } } peers { permanent_uuid: "a51575ac2579457d83f617edc7495a5e" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 44593 } } peers { permanent_uuid: "a89a894deaa14deb8512e35c5c20beca" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 39471 } } }
I20260811 00:22:36.089201 8724 sys_catalog.cc:458] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:22:36.114358 8781 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:22:36.116436 8781 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:22:36.163704 8781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3315839146724e4b9fc2cac80ac54a23: Generated new TSK 0
I20260811 00:22:36.165086 8781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:22:36.762455 8752 catalog_manager.cc:1259] Loaded cluster ID: 25597673d8ec47698e24260fbde48c68
I20260811 00:22:36.762585 8752 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: loading cluster ID for follower catalog manager: success
I20260811 00:22:36.765297 8752 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: acquiring CA information for follower catalog manager: success
I20260811 00:22:36.767506 8752 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a89a894deaa14deb8512e35c5c20beca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:22:36.833761 8776 catalog_manager.cc:1259] Loaded cluster ID: 25597673d8ec47698e24260fbde48c68
I20260811 00:22:36.833899 8776 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: loading cluster ID for follower catalog manager: success
I20260811 00:22:36.837236 8776 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: acquiring CA information for follower catalog manager: success
I20260811 00:22:36.839352 8776 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a51575ac2579457d83f617edc7495a5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:22:37.354641 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 8255
I20260811 00:22:37.356237 8393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:37.470966 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 8255
I20260811 00:22:37.504698 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 8414
I20260811 00:22:37.506086 8537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:37.616420 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 8414
I20260811 00:22:37.643394 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 8565
I20260811 00:22:37.644767 8682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:37.755069 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 8565
2026-08-11T00:22:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5466 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:22:37 dist-test-slave-f7th krb5kdc[8831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:22:37 dist-test-slave-f7th krb5kdc[8831](info): set up 2 sockets
Aug 11 00:22:37 dist-test-slave-f7th krb5kdc[8831](info): commencing operation
krb5kdc: starting...
W20260811 00:22:39.071038 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:22:39 dist-test-slave-f7th krb5kdc[8831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:22:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:22:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:22:39.376485 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42325
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:36639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:42325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:39.749964 8859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:39.750320 8859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:39.750461 8859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:39.758271 8859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:39.758430 8859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:39.758510 8859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:39.758586 8859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:39.758646 8859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:39.769124 8859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:36639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:42325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:42325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:39.770561 8859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:39.771975 8859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:39.784520 8871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:39.784601 8878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:39.784642 8872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:39.785789 8859 server_base.cc:1034] running on GCE node
I20260811 00:22:39.786708 8859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:39.788410 8859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:39.789664 8859 hybrid_clock.cc:648] HybridClock initialized: now 1786407759789593 us; error 62 us; skew 500 ppm
I20260811 00:22:39.790059 8859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:39 dist-test-slave-f7th krb5kdc[8831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407759, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:39.793582 8859 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:22:39.795382 8859 webserver.cc:460] Webserver started at http://127.16.71.190:37909/ using document root <none> and password file <none>
I20260811 00:22:39.796170 8859 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:39.796316 8859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:39.796710 8859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:39.800168 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "00667f1b3b4440febcf5ff2d57a93fea"
format_stamp: "Formatted at 2026-08-11 00:22:39 on dist-test-slave-f7th"
server_key: "89cbe506f8136ce7cd1f3d6089673ce5"
server_key_iv: "932782ca1d5f2258fb9ada6aa113ad1d"
server_key_version: "encryptionkey@0"
I20260811 00:22:39.801138 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "00667f1b3b4440febcf5ff2d57a93fea"
format_stamp: "Formatted at 2026-08-11 00:22:39 on dist-test-slave-f7th"
server_key: "89cbe506f8136ce7cd1f3d6089673ce5"
server_key_iv: "932782ca1d5f2258fb9ada6aa113ad1d"
server_key_version: "encryptionkey@0"
I20260811 00:22:39.807297 8859 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:22:39.811461 8892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:39.813125 8859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:22:39.813401 8859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "00667f1b3b4440febcf5ff2d57a93fea"
format_stamp: "Formatted at 2026-08-11 00:22:39 on dist-test-slave-f7th"
server_key: "89cbe506f8136ce7cd1f3d6089673ce5"
server_key_iv: "932782ca1d5f2258fb9ada6aa113ad1d"
server_key_version: "encryptionkey@0"
I20260811 00:22:39.813640 8859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:39.827241 8859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:39.832285 8859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:39.832657 8859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:39.853279 8859 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:42325
I20260811 00:22:39.853355 8990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:42325 every 8 connection(s)
I20260811 00:22:39.854871 8859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:22:39.859628 8997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:39.859709 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 8859
I20260811 00:22:39.860002 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:22:39.860814 16670 external_mini_cluster.cc:1428] Setting key a3e1cf2cd23946cde735174aa34d16cf
I20260811 00:22:39.874653 8997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea: Bootstrap starting.
Aug 11 00:22:39 dist-test-slave-f7th krb5kdc[8831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:39.878571 8997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:39.879848 8997 log.cc:824] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:39.883260 8997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea: No bootstrap required, opened a new log
I20260811 00:22:39.887327 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:39.863114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60482 (local address 127.16.71.190:42325)
0811 00:22:39.864264 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:39.864278 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:39.864327 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:22:39.867020 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:39.867061 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:39.867619 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:39.868084 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:39.869908 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:39.871502 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:39.873148 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:39.874002 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:39.874223 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:39.879104 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:39.879114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:39.879134 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:39.879185 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:39.882510 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:39.883422 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:39.883432 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:39.883442 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:39.883656 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:39.884132 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:39.884136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:39.884138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:39.884786 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:39.885181 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:39.885688 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 00:22:39.886173 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":539,"threads_started":1}
I20260811 00:22:39.891738 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:22:39.892438 8997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00667f1b3b4440febcf5ff2d57a93fea" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42325 } }
I20260811 00:22:39.892916 8997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:39.893010 8997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00667f1b3b4440febcf5ff2d57a93fea, State: Initialized, Role: FOLLOWER
I20260811 00:22:39.893591 8997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00667f1b3b4440febcf5ff2d57a93fea" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42325 } }
I20260811 00:22:39.893831 8997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:22:39.893975 8997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:22:39.894137 8997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:39.896786 8997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00667f1b3b4440febcf5ff2d57a93fea" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42325 } }
I20260811 00:22:39.897336 8997 leader_election.cc:302] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00667f1b3b4440febcf5ff2d57a93fea; no voters:
I20260811 00:22:39.898012 8997 leader_election.cc:288] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:22:39.898149 9008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:22:39.898854 9008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [term 1 LEADER]: Becoming Leader. State: Replica: 00667f1b3b4440febcf5ff2d57a93fea, State: Running, Role: LEADER
I20260811 00:22:39.899480 9008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00667f1b3b4440febcf5ff2d57a93fea" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42325 } }
I20260811 00:22:39.900393 8997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:22:39.902397 9009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00667f1b3b4440febcf5ff2d57a93fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00667f1b3b4440febcf5ff2d57a93fea" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42325 } } }
I20260811 00:22:39.902868 9009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:39.904400 9018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:22:39.904393 9011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00667f1b3b4440febcf5ff2d57a93fea. Latest consensus state: current_term: 1 leader_uuid: "00667f1b3b4440febcf5ff2d57a93fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00667f1b3b4440febcf5ff2d57a93fea" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 42325 } } }
I20260811 00:22:39.904767 9011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:39.909775 9018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:22:39.919777 9018 catalog_manager.cc:1347] Generated new cluster ID: 94c17c1ef9a443e1b2031bc7798e8258
I20260811 00:22:39.919996 9018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:22:39.936245 9018 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:22:39.937789 9018 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:22:39.947729 9018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00667f1b3b4440febcf5ff2d57a93fea: Generated new TSK 0
I20260811 00:22:39.949026 9018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:22:40.007236 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 8859
I20260811 00:22:40.008817 8983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:40.109613 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 8859
2026-08-11T00:22:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2356 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:22:40 dist-test-slave-f7th krb5kdc[9053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:22:40 dist-test-slave-f7th krb5kdc[9053](info): set up 2 sockets
Aug 11 00:22:40 dist-test-slave-f7th krb5kdc[9053](info): commencing operation
krb5kdc: starting...
W20260811 00:22:41.419909 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:22:41 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:22:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:22:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:22:41.732597 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36785
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:36785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:42.107785 9082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:42.108151 9082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:42.108285 9082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:42.116176 9082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:42.116328 9082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:42.116377 9082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:42.116456 9082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:42.116520 9082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:42.127096 9082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:36785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:42.128607 9082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:42.130072 9082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:42.144018 9091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:42.144018 9089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:42.144375 9088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:42.144984 9082 server_base.cc:1034] running on GCE node
I20260811 00:22:42.145931 9082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:42.147540 9082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:42.148726 9082 hybrid_clock.cc:648] HybridClock initialized: now 1786407762148670 us; error 36 us; skew 500 ppm
I20260811 00:22:42.149143 9082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:42 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407762, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:42.152645 9082 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:22:42.154263 9082 webserver.cc:460] Webserver started at http://127.16.71.190:35755/ using document root <none> and password file <none>
I20260811 00:22:42.154979 9082 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:42.155103 9082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:42.155372 9082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:42.158685 9082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "a6885e0ac8ef45bb91f8413d679637a2"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "0a1862cb07ca4d04f9b90c02ed9d3d39"
server_key_iv: "715b4bfadb84825e1e056b936c1ec38e"
server_key_version: "encryptionkey@0"
I20260811 00:22:42.159476 9082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "a6885e0ac8ef45bb91f8413d679637a2"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "0a1862cb07ca4d04f9b90c02ed9d3d39"
server_key_iv: "715b4bfadb84825e1e056b936c1ec38e"
server_key_version: "encryptionkey@0"
I20260811 00:22:42.165269 9082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 00:22:42.169266 9110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:42.170840 9082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:22:42.171015 9082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "a6885e0ac8ef45bb91f8413d679637a2"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "0a1862cb07ca4d04f9b90c02ed9d3d39"
server_key_iv: "715b4bfadb84825e1e056b936c1ec38e"
server_key_version: "encryptionkey@0"
I20260811 00:22:42.171201 9082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:42.189572 9082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:42.194046 9082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:42.194314 9082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:42.213406 9082 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:36785
I20260811 00:22:42.213459 9195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:36785 every 8 connection(s)
I20260811 00:22:42.214977 9082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:22:42.219302 9197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:42.224377 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 9082
I20260811 00:22:42.224602 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:22:42.225467 16670 external_mini_cluster.cc:1428] Setting key 203248e12de0672ed3932628c7b71713
I20260811 00:22:42.232263 9197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Bootstrap starting.
I20260811 00:22:42.236868 9197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:42.238325 9197 log.cc:824] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:22:42 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:42.242339 9197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: No bootstrap required, opened a new log
I20260811 00:22:42.253161 9197 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:42.253703 9197 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:42.253824 9197 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6885e0ac8ef45bb91f8413d679637a2, State: Initialized, Role: FOLLOWER
I20260811 00:22:42.254192 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:42.227715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.16.71.190:36785)
0811 00:22:42.228651 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:42.228668 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:42.228719 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:22:42.230632 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:42.230663 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:42.231155 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:42.231548 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:42.233635 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:42.235157 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:42.237272 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:42.238122 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:42.238284 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:42.243830 (+ 5546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:42.243840 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:42.243860 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:42.243907 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:42.248443 (+ 4536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:42.249581 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:42.249593 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:42.249607 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:42.249890 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:42.250328 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:42.250334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:42.250338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:42.251266 (+ 928us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:42.251634 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:42.252055 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 00:22:42.252599 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":414,"threads_started":1}
I20260811 00:22:42.254611 9197 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:42.254866 9197 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:22:42.255003 9197 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:22:42.255175 9197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:42.257676 9197 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:42.258204 9197 leader_election.cc:302] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6885e0ac8ef45bb91f8413d679637a2; no voters:
I20260811 00:22:42.258831 9197 leader_election.cc:288] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:22:42.258976 9205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:22:42.259690 9205 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 LEADER]: Becoming Leader. State: Replica: a6885e0ac8ef45bb91f8413d679637a2, State: Running, Role: LEADER
I20260811 00:22:42.260267 9205 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:42.261144 9197 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:22:42.263347 9206 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6885e0ac8ef45bb91f8413d679637a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } } }
I20260811 00:22:42.263777 9206 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:42.263389 9208 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6885e0ac8ef45bb91f8413d679637a2. Latest consensus state: current_term: 1 leader_uuid: "a6885e0ac8ef45bb91f8413d679637a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } } }
I20260811 00:22:42.264031 9208 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:42.267105 9226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:22:42.272084 9226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:22:42.281886 9226 catalog_manager.cc:1347] Generated new cluster ID: e778653fa6594d8fa2b1a9b921fe9efb
I20260811 00:22:42.282025 9226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:22:42.295750 9226 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:22:42.297837 9226 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:22:42.313802 9226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Generated new TSK 0
I20260811 00:22:42.314908 9226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:22:42.396682 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:36785
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:42.771426 9250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:42.771747 9250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:42.771873 9250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:42.779870 9250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:42.780015 9250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:42.780169 9250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:22:42.791102 9250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:42.792584 9250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:42.794142 9250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:42.809911 9258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:42.809865 9263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:42.809933 9257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:42.811278 9250 server_base.cc:1034] running on GCE node
I20260811 00:22:42.812006 9250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:42.813513 9250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:42.814973 9250 hybrid_clock.cc:648] HybridClock initialized: now 1786407762814902 us; error 60 us; skew 500 ppm
I20260811 00:22:42.815388 9250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:42 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407762, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:42.818974 9250 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:22:42.820775 9250 webserver.cc:460] Webserver started at http://127.16.71.129:33947/ using document root <none> and password file <none>
I20260811 00:22:42.821518 9250 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:42.821632 9250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:42.821918 9250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:42.825204 9250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "7a5da02d8eb39302cc5045a9aae61e8a"
server_key_iv: "04f0d18a0e885e548ff70e3a6556b769"
server_key_version: "encryptionkey@0"
I20260811 00:22:42.826087 9250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "7a5da02d8eb39302cc5045a9aae61e8a"
server_key_iv: "04f0d18a0e885e548ff70e3a6556b769"
server_key_version: "encryptionkey@0"
I20260811 00:22:42.832765 9250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:22:42.837018 9273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:42.838724 9250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:22:42.838922 9250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "7a5da02d8eb39302cc5045a9aae61e8a"
server_key_iv: "04f0d18a0e885e548ff70e3a6556b769"
server_key_version: "encryptionkey@0"
I20260811 00:22:42.839182 9250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:42.877064 9250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:42.883927 9250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:42.884351 9250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:42.885713 9250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:42.887678 9250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:42.887787 9250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:42.887930 9250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:42.888011 9250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:42.942190 9250 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:45615
I20260811 00:22:42.942301 9476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:45615 every 8 connection(s)
I20260811 00:22:42.943874 9250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:22:42.952184 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 9250
I20260811 00:22:42.952440 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:22:42.953435 16670 external_mini_cluster.cc:1428] Setting key 50778a07a499b928e67a6f8380cc34a0
Aug 11 00:22:42 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407762, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:42.969893 9479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:42.947533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:37953 (local address 127.16.71.190:36785)
0811 00:22:42.948107 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:42.948112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:42.949448 (+ 1336us) server_negotiation.cc:407] Connection header received
0811 00:22:42.951844 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:42.951851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:42.951994 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:42.952117 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:42.954245 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:42.955212 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:42.957616 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:42.958079 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:42.958220 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:42.962349 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:42.962356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:42.962359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:42.962399 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:42.965571 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:42.967047 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:42.967053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:42.967056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:42.967216 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:42.967763 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:42.967768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:42.967770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:42.968082 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:42.968241 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:42.969162 (+ 921us) server_negotiation.cc:299] Negotiation successful
0811 00:22:42.969498 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":327,"threads_started":1}
I20260811 00:22:42.971242 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:42.948197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:37953)
0811 00:22:42.949201 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:42.949264 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:42.951287 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:42.952498 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:42.952512 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:42.953263 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:42.954016 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:42.954040 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:42.955420 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:42.955424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:42.957403 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:42.957424 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:42.958384 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:42.958393 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:42.958679 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:42.959465 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:42.959494 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:42.962046 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:42.965917 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:42.965948 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:42.966104 (+ 156us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:42.966830 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:42.967350 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:42.967356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:42.967360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:42.967616 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:42.968238 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:42.968248 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:42.968704 (+ 456us) client_negotiation.cc:769] Sending connection context
0811 00:22:42.969022 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 00:22:42.969735 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":436,"threads_started":1}
I20260811 00:22:42.973099 9477 heartbeater.cc:347] Connected to a master server at 127.16.71.190:36785
I20260811 00:22:42.973440 9477 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:42.974298 9477 heartbeater.cc:511] Master 127.16.71.190:36785 requested a full tablet report, sending...
I20260811 00:22:42.977159 9146 ts_manager.cc:194] Registered new tserver with Master: 0896a57ff3a64cf4bd7ee2c07c7a9ab7 (127.16.71.129:45615)
I20260811 00:22:42.979116 9146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:37953
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:22:43.029068 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:36785
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:43.397871 9486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:43.398205 9486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:43.398339 9486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:43.406008 9486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:43.406133 9486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:43.406278 9486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:22:43.416595 9486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:43.418104 9486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:43.419500 9486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:43.433743 9497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:43.433748 9495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:43.433825 9500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:43.434525 9486 server_base.cc:1034] running on GCE node
I20260811 00:22:43.435235 9486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:43.436332 9486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:43.437588 9486 hybrid_clock.cc:648] HybridClock initialized: now 1786407763437551 us; error 98 us; skew 500 ppm
I20260811 00:22:43.437981 9486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:43 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407763, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:43.441699 9486 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:22:43.443627 9486 webserver.cc:460] Webserver started at http://127.16.71.130:45505/ using document root <none> and password file <none>
I20260811 00:22:43.444510 9486 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:43.444693 9486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:43.445056 9486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:43.448467 9486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "1db7f497d1514eb0ac71ac4a8b2d8289"
format_stamp: "Formatted at 2026-08-11 00:22:43 on dist-test-slave-f7th"
server_key: "17fd754990c94d2ccc4bf3554887f561"
server_key_iv: "37c0e9732198bb54daa5967de44db1a0"
server_key_version: "encryptionkey@0"
I20260811 00:22:43.449545 9486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "1db7f497d1514eb0ac71ac4a8b2d8289"
format_stamp: "Formatted at 2026-08-11 00:22:43 on dist-test-slave-f7th"
server_key: "17fd754990c94d2ccc4bf3554887f561"
server_key_iv: "37c0e9732198bb54daa5967de44db1a0"
server_key_version: "encryptionkey@0"
I20260811 00:22:43.455564 9486 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:22:43.460021 9511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:43.461917 9486 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 00:22:43.462157 9486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "1db7f497d1514eb0ac71ac4a8b2d8289"
format_stamp: "Formatted at 2026-08-11 00:22:43 on dist-test-slave-f7th"
server_key: "17fd754990c94d2ccc4bf3554887f561"
server_key_iv: "37c0e9732198bb54daa5967de44db1a0"
server_key_version: "encryptionkey@0"
I20260811 00:22:43.462393 9486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:43.482738 9486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:43.487655 9486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:43.487967 9486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:43.489100 9486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:43.490796 9486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:43.490887 9486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:43.491006 9486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:43.491062 9486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:43.524554 9486 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:38641
I20260811 00:22:43.524600 9718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:38641 every 8 connection(s)
I20260811 00:22:43.526008 9486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:22:43.530651 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 9486
I20260811 00:22:43.530835 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:22:43.531256 16670 external_mini_cluster.cc:1428] Setting key 3dd75f63bae36706e661d97f62addf4b
Aug 11 00:22:43 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407763, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:43.548105 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:43.528911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:60097 (local address 127.16.71.190:36785)
0811 00:22:43.529529 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:43.529534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:43.530369 (+ 835us) server_negotiation.cc:407] Connection header received
0811 00:22:43.532428 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:43.532434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:43.532519 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:43.532647 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:43.534466 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:43.535240 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:43.536986 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:43.537449 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:43.537570 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:43.541647 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:43.541653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:43.541657 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:43.541707 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:43.544589 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:43.545472 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:43.545478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:43.545480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:43.545619 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:43.546113 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:43.546117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:43.546119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:43.546439 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:43.546586 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:43.547453 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 00:22:43.547720 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":272,"threads_started":1}
I20260811 00:22:43.549223 9729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:43.529349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:60097)
0811 00:22:43.530154 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:43.530199 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:43.532130 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:43.532940 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:43.532950 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:43.533634 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:43.534278 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:43.534298 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:43.535402 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:43.535406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:43.536804 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:43.536822 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:43.537715 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:43.537723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:43.537978 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:43.538734 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:43.538767 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:43.541354 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:43.544793 (+ 3439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:43.544803 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:43.544825 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:43.545317 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:43.545742 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:43.545747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:43.545751 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:43.545978 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:43.546584 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:43.546594 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:43.547015 (+ 421us) client_negotiation.cc:769] Sending connection context
0811 00:22:43.547293 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 00:22:43.547913 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":290,"threads_started":1}
I20260811 00:22:43.550899 9721 heartbeater.cc:347] Connected to a master server at 127.16.71.190:36785
I20260811 00:22:43.551218 9721 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:43.552086 9721 heartbeater.cc:511] Master 127.16.71.190:36785 requested a full tablet report, sending...
I20260811 00:22:43.553822 9146 ts_manager.cc:194] Registered new tserver with Master: 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641)
I20260811 00:22:43.555002 9146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:60097
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:22:43.596750 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:36785
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:43.958806 9739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:43.959173 9739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:43.959327 9739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:43.967703 9739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:43.967856 9739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:43.968032 9739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:22:43.979988 9739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:43.981518 9739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:43.982725 9477 heartbeater.cc:503] Master 127.16.71.190:36785 was elected leader, sending a full tablet report...
I20260811 00:22:43.983100 9739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:43.998488 9751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:43.998347 9755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:43.998436 9749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:43.999349 9739 server_base.cc:1034] running on GCE node
I20260811 00:22:44.000118 9739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:44.001252 9739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:44.002534 9739 hybrid_clock.cc:648] HybridClock initialized: now 1786407764002477 us; error 63 us; skew 500 ppm
I20260811 00:22:44.002983 9739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:44 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407764, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:44.007189 9739 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:22:44.008925 9739 webserver.cc:460] Webserver started at http://127.16.71.131:36147/ using document root <none> and password file <none>
I20260811 00:22:44.009752 9739 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:44.009879 9739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:44.010206 9739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:22:44.013597 9739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "753ddd2142b24d35bb02ea5ae47d2794"
format_stamp: "Formatted at 2026-08-11 00:22:44 on dist-test-slave-f7th"
server_key: "7c9afd3bd92a9c45ebb15c3a2765f8e0"
server_key_iv: "615bfc746f499b59d0bbba282f9ef831"
server_key_version: "encryptionkey@0"
I20260811 00:22:44.014575 9739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "753ddd2142b24d35bb02ea5ae47d2794"
format_stamp: "Formatted at 2026-08-11 00:22:44 on dist-test-slave-f7th"
server_key: "7c9afd3bd92a9c45ebb15c3a2765f8e0"
server_key_iv: "615bfc746f499b59d0bbba282f9ef831"
server_key_version: "encryptionkey@0"
I20260811 00:22:44.020648 9739 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:22:44.024853 9766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:44.026439 9739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:22:44.026639 9739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "753ddd2142b24d35bb02ea5ae47d2794"
format_stamp: "Formatted at 2026-08-11 00:22:44 on dist-test-slave-f7th"
server_key: "7c9afd3bd92a9c45ebb15c3a2765f8e0"
server_key_iv: "615bfc746f499b59d0bbba282f9ef831"
server_key_version: "encryptionkey@0"
I20260811 00:22:44.026870 9739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:44.063884 9739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:44.068593 9739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:44.068922 9739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:44.070017 9739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:44.071497 9739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:44.071588 9739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:44.071722 9739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:44.071777 9739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:44.104154 9739 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:45973
I20260811 00:22:44.104235 9988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:45973 every 8 connection(s)
I20260811 00:22:44.105597 9739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:22:44.109813 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 9739
I20260811 00:22:44.110036 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:22:44.110615 16670 external_mini_cluster.cc:1428] Setting key 56b0d711f300b66fc19b76100d4fd2ca
Aug 11 00:22:44 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407764, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:44.128835 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.108389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:42539 (local address 127.16.71.190:36785)
0811 00:22:44.108876 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.108881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.110096 (+ 1215us) server_negotiation.cc:407] Connection header received
0811 00:22:44.112419 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.112425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.112535 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.112677 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:44.114832 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.115626 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.117376 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.117807 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.117918 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.121779 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:44.121785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:44.121789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:44.121827 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:44.124607 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.125879 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.125884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.125887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.126196 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.126758 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.126763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.126765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.127120 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:44.127282 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.128129 (+ 847us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.128404 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":255,"threads_started":1}
I20260811 00:22:44.129871 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.108905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:42539)
0811 00:22:44.109785 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:44.109844 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:44.112096 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:44.113018 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:44.113029 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:44.113767 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:44.114618 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.114648 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.115768 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.115772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.117175 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.117190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.118051 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.118057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.118274 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.118957 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:44.119002 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:44.121495 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:44.124823 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:44.124835 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:44.124857 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:44.125697 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:44.126361 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:44.126365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:44.126368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:44.126617 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:44.127276 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:44.127286 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:44.127703 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 00:22:44.127990 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 00:22:44.128637 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":323,"threads_started":1}
I20260811 00:22:44.131603 9989 heartbeater.cc:347] Connected to a master server at 127.16.71.190:36785
I20260811 00:22:44.131925 9989 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:44.132764 9989 heartbeater.cc:511] Master 127.16.71.190:36785 requested a full tablet report, sending...
I20260811 00:22:44.134306 9146 ts_manager.cc:194] Registered new tserver with Master: 753ddd2142b24d35bb02ea5ae47d2794 (127.16.71.131:45973)
I20260811 00:22:44.135202 9146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:42539
I20260811 00:22:44.140105 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:22:44.159720 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.144931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.16.71.190:36785)
0811 00:22:44.145144 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.145151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.145690 (+ 539us) server_negotiation.cc:407] Connection header received
0811 00:22:44.145814 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.145818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.145892 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.146118 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:44.147864 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.148858 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.150402 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.150857 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.150975 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.153781 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:44.153788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:44.153791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:44.153836 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:44.156729 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.157640 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.157644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.157647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.157793 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.158210 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.158214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.158216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.158594 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:44.158789 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.159190 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.159402 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:22:44.169157 9146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:22:44.176780 9146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:22:44.211283 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.192516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.16.71.131:45973)
0811 00:22:44.201711 (+ 9195us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.201723 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.201769 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:22:44.201907 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.201918 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.202273 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.202559 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:44.203654 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.205371 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.208365 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.209941 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.210091 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.210362 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.210726 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.210919 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8958,"thread_start_us":8774,"threads_started":1}
I20260811 00:22:44.211328 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.191823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:45973 (local address 127.0.0.1:44848)
0811 00:22:44.195310 (+ 3487us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:44.195340 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:44.195484 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:44.202634 (+ 7150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:44.202645 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:44.202689 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:44.203325 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.203345 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.205530 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.205536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.208139 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.208157 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.210186 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.210193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.210439 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.210507 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 00:22:44.210648 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 00:22:44.210869 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3335,"thread_start_us":11194,"threads_started":1}
I20260811 00:22:44.212037 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.191724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:45615 (local address 127.0.0.1:48736)
0811 00:22:44.193295 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:44.193368 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:44.193596 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:44.204765 (+ 11169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:44.204770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:44.204797 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:44.205155 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.205165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.207136 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.207142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.209187 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.209202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.211158 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.211164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.211326 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.211357 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:22:44.211429 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 00:22:44.211600 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1263,"mutex_wait_us":2,"thread_start_us":1075,"threads_started":1}
I20260811 00:22:44.212369 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.195228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.16.71.130:38641)
0811 00:22:44.195988 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.195999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.203543 (+ 7544us) server_negotiation.cc:407] Connection header received
0811 00:22:44.203694 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.203704 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.204012 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.204252 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:44.204929 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.206340 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.208932 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.210073 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.210223 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.210425 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.211877 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.212077 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":459,"threads_started":1}
I20260811 00:22:44.212883 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.194608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:38641 (local address 127.0.0.1:52382)
0811 00:22:44.203378 (+ 8770us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:44.203396 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:44.203542 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:44.204265 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:44.204271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:44.204303 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:44.204711 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.204725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.206531 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.206536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.208735 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.208753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.210563 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.210571 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.210757 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.210793 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:22:44.210882 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:22:44.211056 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8601,"mutex_wait_us":77,"thread_start_us":791,"threads_started":1}
I20260811 00:22:44.213820 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.191923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.16.71.129:45615)
0811 00:22:44.204168 (+ 12245us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.204177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.204208 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:22:44.204302 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.204308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.204552 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.204765 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:44.205331 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.206958 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.209386 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.210899 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.212906 (+ 2007us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.213191 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.213274 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.213497 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12045,"thread_start_us":11883,"threads_started":1}
I20260811 00:22:44.215962 9861 tablet_service.cc:1467] Processing CreateTablet for tablet c898cce514ea45478084e7f53c8c12d7 (DEFAULT_TABLE table=test-table [id=21e620b682534faa9cc835641268f6f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:44.217790 9861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c898cce514ea45478084e7f53c8c12d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:44.218585 9359 tablet_service.cc:1467] Processing CreateTablet for tablet c898cce514ea45478084e7f53c8c12d7 (DEFAULT_TABLE table=test-table [id=21e620b682534faa9cc835641268f6f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:44.217180 9597 tablet_service.cc:1467] Processing CreateTablet for tablet c898cce514ea45478084e7f53c8c12d7 (DEFAULT_TABLE table=test-table [id=21e620b682534faa9cc835641268f6f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:44.220189 9359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c898cce514ea45478084e7f53c8c12d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:44.220211 9597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c898cce514ea45478084e7f53c8c12d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:44.235174 10016 tablet_bootstrap.cc:492] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Bootstrap starting.
I20260811 00:22:44.237465 10017 tablet_bootstrap.cc:492] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Bootstrap starting.
I20260811 00:22:44.239218 10016 tablet_bootstrap.cc:654] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:44.239908 10019 tablet_bootstrap.cc:492] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Bootstrap starting.
I20260811 00:22:44.240671 10016 log.cc:824] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:44.241379 10017 tablet_bootstrap.cc:654] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:44.242672 10017 log.cc:824] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:44.243721 10019 tablet_bootstrap.cc:654] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:44.244603 10016 tablet_bootstrap.cc:492] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: No bootstrap required, opened a new log
I20260811 00:22:44.244948 10016 ts_tablet_manager.cc:1397] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.004s
I20260811 00:22:44.245056 10019 log.cc:824] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:44.246470 10017 tablet_bootstrap.cc:492] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: No bootstrap required, opened a new log
I20260811 00:22:44.246776 10017 ts_tablet_manager.cc:1397] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 00:22:44.251153 10019 tablet_bootstrap.cc:492] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: No bootstrap required, opened a new log
I20260811 00:22:44.251464 10019 ts_tablet_manager.cc:1397] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 00:22:44.256028 10016 raft_consensus.cc:348] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.256608 10016 raft_consensus.cc:374] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:44.256778 10016 raft_consensus.cc:729] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753ddd2142b24d35bb02ea5ae47d2794, State: Initialized, Role: FOLLOWER
I20260811 00:22:44.257535 10016 consensus_queue.cc:260] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.257534 10017 raft_consensus.cc:348] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.258013 10017 raft_consensus.cc:374] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:44.258131 10017 raft_consensus.cc:729] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0896a57ff3a64cf4bd7ee2c07c7a9ab7, State: Initialized, Role: FOLLOWER
I20260811 00:22:44.258913 9989 heartbeater.cc:503] Master 127.16.71.190:36785 was elected leader, sending a full tablet report...
I20260811 00:22:44.258888 10017 consensus_queue.cc:260] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.259331 10019 raft_consensus.cc:348] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.259785 10019 raft_consensus.cc:374] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:44.259886 10019 raft_consensus.cc:729] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1db7f497d1514eb0ac71ac4a8b2d8289, State: Initialized, Role: FOLLOWER
I20260811 00:22:44.260522 10019 consensus_queue.cc:260] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.262266 10016 ts_tablet_manager.cc:1428] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260811 00:22:44.265735 10017 ts_tablet_manager.cc:1428] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Time spent starting tablet: real 0.019s user 0.005s sys 0.013s
I20260811 00:22:44.267165 9721 heartbeater.cc:503] Master 127.16.71.190:36785 was elected leader, sending a full tablet report...
I20260811 00:22:44.267884 10019 ts_tablet_manager.cc:1428] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Time spent starting tablet: real 0.016s user 0.013s sys 0.005s
W20260811 00:22:44.278110 9723 tablet.cc:2369] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:22:44.357390 9993 tablet.cc:2369] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:22:44.447264 9478 tablet.cc:2369] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:22:44.495685 10026 raft_consensus.cc:482] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:22:44.496028 10026 raft_consensus.cc:504] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.497992 10026 leader_election.cc:288] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753ddd2142b24d35bb02ea5ae47d2794 (127.16.71.131:45973), 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641)
I20260811 00:22:44.505589 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.498406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:59049 (local address 127.16.71.131:45973)
0811 00:22:44.498687 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.498694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.499631 (+ 937us) server_negotiation.cc:407] Connection header received
0811 00:22:44.499828 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.499833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.499899 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.499969 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:44.500668 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.501473 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.503522 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.504383 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.504535 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.504598 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.505160 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.505310 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 00:22:44.505632 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.498776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:52037 (local address 127.16.71.130:38641)
0811 00:22:44.499013 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.499020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.499704 (+ 684us) server_negotiation.cc:407] Connection header received
0811 00:22:44.499857 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.499861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.499921 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.500012 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:44.500678 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.501391 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.503526 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.504553 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.504778 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.504855 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.505215 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.505371 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 00:22:44.505749 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.498656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:38641 (local address 127.16.71.129:52037)
0811 00:22:44.499561 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:44.499579 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:44.499724 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:44.500046 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:44.500050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:44.500071 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:44.500487 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.500497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.501536 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.501542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.503314 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.503332 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.504841 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.504848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.505049 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.505079 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:22:44.505175 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:22:44.505309 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":126,"thread_start_us":575,"threads_started":1}
I20260811 00:22:44.506170 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.498473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:45973 (local address 127.16.71.129:59049)
0811 00:22:44.499470 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:44.499496 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:44.499669 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:44.500026 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:44.500030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:44.500055 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:44.500481 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.500492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.501616 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.501619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.503317 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:44.503332 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.504690 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.504698 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:44.504949 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.504993 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:22:44.505104 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 00:22:44.505258 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":105,"thread_start_us":530,"threads_started":1}
I20260811 00:22:44.507009 9637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c898cce514ea45478084e7f53c8c12d7" candidate_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" is_pre_election: true
I20260811 00:22:44.507169 9900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c898cce514ea45478084e7f53c8c12d7" candidate_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753ddd2142b24d35bb02ea5ae47d2794" is_pre_election: true
I20260811 00:22:44.507563 9637 raft_consensus.cc:2454] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0896a57ff3a64cf4bd7ee2c07c7a9ab7 in term 0.
I20260811 00:22:44.507668 9900 raft_consensus.cc:2454] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0896a57ff3a64cf4bd7ee2c07c7a9ab7 in term 0.
I20260811 00:22:44.508255 9275 leader_election.cc:302] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0896a57ff3a64cf4bd7ee2c07c7a9ab7, 1db7f497d1514eb0ac71ac4a8b2d8289; no voters:
I20260811 00:22:44.508725 10026 raft_consensus.cc:2781] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:22:44.508895 10026 raft_consensus.cc:482] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:22:44.509003 10026 raft_consensus.cc:3037] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:44.511286 10026 raft_consensus.cc:504] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.511961 10026 leader_election.cc:288] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [CANDIDATE]: Term 1 election: Requested vote from peers 753ddd2142b24d35bb02ea5ae47d2794 (127.16.71.131:45973), 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641)
I20260811 00:22:44.512609 9637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c898cce514ea45478084e7f53c8c12d7" candidate_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289"
I20260811 00:22:44.512648 9900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c898cce514ea45478084e7f53c8c12d7" candidate_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753ddd2142b24d35bb02ea5ae47d2794"
I20260811 00:22:44.512859 9637 raft_consensus.cc:3037] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:44.512878 9900 raft_consensus.cc:3037] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:44.515267 9637 raft_consensus.cc:2454] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0896a57ff3a64cf4bd7ee2c07c7a9ab7 in term 1.
I20260811 00:22:44.515285 9900 raft_consensus.cc:2454] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0896a57ff3a64cf4bd7ee2c07c7a9ab7 in term 1.
I20260811 00:22:44.515801 9275 leader_election.cc:302] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0896a57ff3a64cf4bd7ee2c07c7a9ab7, 1db7f497d1514eb0ac71ac4a8b2d8289; no voters:
I20260811 00:22:44.516156 10026 raft_consensus.cc:2781] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:22:44.516695 10026 raft_consensus.cc:686] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 LEADER]: Becoming Leader. State: Replica: 0896a57ff3a64cf4bd7ee2c07c7a9ab7, State: Running, Role: LEADER
I20260811 00:22:44.517234 10026 consensus_queue.cc:237] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } }
I20260811 00:22:44.522312 9146 catalog_manager.cc:5495] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0896a57ff3a64cf4bd7ee2c07c7a9ab7 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260811 00:22:44.579154 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.573307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.16.71.129:45615)
0811 00:22:44.573536 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.573542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.573563 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:22:44.573827 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.573832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.573942 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.574096 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:22:44.574536 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.575316 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.576877 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.577300 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.577436 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.577969 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:22:44.578520 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:22:44.578664 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.578781 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.578874 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:22:44.588243 9637 raft_consensus.cc:1261] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Refusing update from remote peer 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:44.588243 9900 raft_consensus.cc:1261] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 FOLLOWER]: Refusing update from remote peer 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:44.589359 10037 consensus_queue.cc:1035] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:22:44.589742 10026 consensus_queue.cc:1035] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:22:44.599481 10043 mvcc.cc:205] Tried to move back new op lower bound from 7317126203742453760 to 7317126203466514432. Current Snapshot: MvccSnapshot[applied={T|T < 7317126203742453760}]
I20260811 00:22:44.618480 9146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:22:44.618755 9146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:22:44.623070 9146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Sending DeleteTablet for 3 replicas of tablet c898cce514ea45478084e7f53c8c12d7
I20260811 00:22:44.624461 9359 tablet_service.cc:1514] Processing DeleteTablet for tablet c898cce514ea45478084e7f53c8c12d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:44 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:48736
I20260811 00:22:44.624433 9597 tablet_service.cc:1514] Processing DeleteTablet for tablet c898cce514ea45478084e7f53c8c12d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:44 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:52382
I20260811 00:22:44.624496 9861 tablet_service.cc:1514] Processing DeleteTablet for tablet c898cce514ea45478084e7f53c8c12d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:44 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:44848
I20260811 00:22:44.626492 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 9250
I20260811 00:22:44.637863 10055 tablet_replica.cc:330] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: stopping tablet replica
I20260811 00:22:44.638689 10056 tablet_replica.cc:330] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: stopping tablet replica
I20260811 00:22:44.639515 10056 raft_consensus.cc:2229] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:22:44.639966 10054 tablet_replica.cc:330] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: stopping tablet replica
I20260811 00:22:44.640678 10054 raft_consensus.cc:2229] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:22:44.640142 10055 raft_consensus.cc:2229] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:22:44.641335 10054 raft_consensus.cc:2258] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:44.641736 10055 raft_consensus.cc:2258] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:44.640756 10056 raft_consensus.cc:2258] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:44.644398 10055 ts_tablet_manager.cc:1905] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:44.644403 10054 ts_tablet_manager.cc:1905] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:44.649823 10056 ts_tablet_manager.cc:1905] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:22:44 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:22:44.652843 10054 ts_tablet_manager.cc:1918] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:44.652853 10055 ts_tablet_manager.cc:1918] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:44.653100 10054 log.cc:1196] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/c898cce514ea45478084e7f53c8c12d7
I20260811 00:22:44.653218 10055 log.cc:1196] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/c898cce514ea45478084e7f53c8c12d7
I20260811 00:22:44.653825 10054 ts_tablet_manager.cc:1939] T c898cce514ea45478084e7f53c8c12d7 P 1db7f497d1514eb0ac71ac4a8b2d8289: Deleting consensus metadata
I20260811 00:22:44.653904 10055 ts_tablet_manager.cc:1939] T c898cce514ea45478084e7f53c8c12d7 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Deleting consensus metadata
I20260811 00:22:44.656257 9116 catalog_manager.cc:4841] TS 0896a57ff3a64cf4bd7ee2c07c7a9ab7 (127.16.71.129:45615): tablet c898cce514ea45478084e7f53c8c12d7 (table test-table [id=21e620b682534faa9cc835641268f6f4]) successfully deleted
I20260811 00:22:44.657114 9113 catalog_manager.cc:4841] TS 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641): tablet c898cce514ea45478084e7f53c8c12d7 (table test-table [id=21e620b682534faa9cc835641268f6f4]) successfully deleted
I20260811 00:22:44.660264 10056 ts_tablet_manager.cc:1918] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:44.660502 10056 log.cc:1196] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/c898cce514ea45478084e7f53c8c12d7
I20260811 00:22:44.661339 10056 ts_tablet_manager.cc:1939] T c898cce514ea45478084e7f53c8c12d7 P 753ddd2142b24d35bb02ea5ae47d2794: Deleting consensus metadata
I20260811 00:22:44.661495 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.627946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.16.71.129:45615)
0811 00:22:44.628158 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.628165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.639165 (+ 11000us) server_negotiation.cc:407] Connection header received
0811 00:22:44.639436 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.639442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.639584 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.639713 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:44.642069 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.643360 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.645652 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.646373 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.646504 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.652755 (+ 6251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:44.652778 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:44.652799 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:44.652875 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:44.657809 (+ 4934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.658718 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.658727 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.658730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.658918 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.659382 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.659386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.659388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.659800 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:44.660433 (+ 633us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.660657 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.660945 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:22:44.662290 9465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:44.663462 9123 catalog_manager.cc:4841] TS 753ddd2142b24d35bb02ea5ae47d2794 (127.16.71.131:45973): tablet c898cce514ea45478084e7f53c8c12d7 (table test-table [id=21e620b682534faa9cc835641268f6f4]) successfully deleted
I20260811 00:22:44.763661 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 9250
I20260811 00:22:44.794941 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 9486
Aug 11 00:22:44 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:22:44.812132 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.795706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.16.71.130:38641)
0811 00:22:44.795897 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.795904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.795925 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:22:44.796158 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.796163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.796229 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.796329 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:44.797795 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.798578 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.800070 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.800512 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.800664 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.805638 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:44.805658 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:44.805676 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:44.805721 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:44.808841 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.809744 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.809753 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.809755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.809949 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.810385 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.810389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.810392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.810846 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:44.811161 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.811503 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.811767 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:22:44.813076 9712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:44.905500 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 9486
I20260811 00:22:44.931927 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 9739
Aug 11 00:22:44 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:22:44.950218 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:44.932760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.16.71.131:45973)
0811 00:22:44.932943 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:44.932948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:44.932971 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:22:44.933127 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:44.933131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:44.933212 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:44.933300 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:44.934730 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.935548 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.937178 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:44.937774 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:44.937935 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:44.942922 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:44.942939 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:44.942958 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:44.943018 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:44.946587 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.947590 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.947599 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.947602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.947783 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:44.948332 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:44.948337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:44.948339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:44.948902 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:44.949179 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:44.949587 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 00:22:44.949802 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:22:44.950989 9975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:45.042543 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 9739
I20260811 00:22:45.069730 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 9082
I20260811 00:22:45.071149 9189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:45.186647 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 9082
I20260811 00:22:45.223308 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36785
--webserver_interface=127.16.71.190
--webserver_port=35755
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:36785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:45.598645 10064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:45.599013 10064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:45.599169 10064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:45.606837 10064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:45.607007 10064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:45.607092 10064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:45.607172 10064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:45.607236 10064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:45.617854 10064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:36785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=35755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:45.619467 10064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:45.620918 10064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:45.650810 10088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:45.650810 10085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:45.650861 10084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:45.652693 10064 server_base.cc:1034] running on GCE node
I20260811 00:22:45.653851 10064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:45.655879 10064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:45.657155 10064 hybrid_clock.cc:648] HybridClock initialized: now 1786407765657080 us; error 66 us; skew 500 ppm
I20260811 00:22:45.657647 10064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:45 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407765, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:45.661785 10064 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:22:45.663887 10064 webserver.cc:460] Webserver started at http://127.16.71.190:35755/ using document root <none> and password file <none>
I20260811 00:22:45.664891 10064 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:45.665086 10064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:45.672166 10064 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 00:22:45.675892 10107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:45.677654 10064 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 00:22:45.677886 10064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "a6885e0ac8ef45bb91f8413d679637a2"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "0a1862cb07ca4d04f9b90c02ed9d3d39"
server_key_iv: "715b4bfadb84825e1e056b936c1ec38e"
server_key_version: "encryptionkey@0"
I20260811 00:22:45.679112 10064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:45.714356 10064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:45.719298 10064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:45.719614 10064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:45.740747 10064 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:36785
I20260811 00:22:45.740772 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:36785 every 8 connection(s)
I20260811 00:22:45.742585 10064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:22:45.746279 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 10064
I20260811 00:22:45.747177 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:45615
--local_ip_for_outbound_sockets=127.16.71.129
--tserver_master_addrs=127.16.71.190:36785
--webserver_port=33947
--webserver_interface=127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:22:45.750833 10201 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:22:45.754668 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Bootstrap starting.
I20260811 00:22:45.776484 10201 log.cc:824] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:45.791594 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:22:45.792207 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Bootstrap complete.
I20260811 00:22:45.802114 10201 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:45.803090 10201 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6885e0ac8ef45bb91f8413d679637a2, State: Initialized, Role: FOLLOWER
I20260811 00:22:45.803819 10201 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:45.804103 10201 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:22:45.804248 10201 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:22:45.804428 10201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:22:45.808926 10201 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:45.809597 10201 leader_election.cc:302] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6885e0ac8ef45bb91f8413d679637a2; no voters:
I20260811 00:22:45.810287 10201 leader_election.cc:288] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:22:45.810459 10210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:22:45.811683 10210 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 LEADER]: Becoming Leader. State: Replica: a6885e0ac8ef45bb91f8413d679637a2, State: Running, Role: LEADER
I20260811 00:22:45.812394 10210 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:45.812800 10201 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:22:45.815744 10215 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6885e0ac8ef45bb91f8413d679637a2. Latest consensus state: current_term: 2 leader_uuid: "a6885e0ac8ef45bb91f8413d679637a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } } }
I20260811 00:22:45.816241 10215 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:45.815623 10214 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a6885e0ac8ef45bb91f8413d679637a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } } }
I20260811 00:22:45.816790 10214 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:45.817723 10227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:22:45.835901 10227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:22:45.837946 10227 catalog_manager.cc:1259] Loaded cluster ID: e778653fa6594d8fa2b1a9b921fe9efb
I20260811 00:22:45.838193 10227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:22:45.842532 10227 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 00:22:45.863905 10227 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 00:22:45.887707 10227 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:22:45.889855 10227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Loaded TSK: 0
I20260811 00:22:45.891664 10227 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:22:46.152486 10205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:46.152868 10205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:46.153012 10205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:46.160912 10205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:46.161068 10205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:46.161222 10205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:22:46.172036 10205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:45615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=33947
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:46.173584 10205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:46.175114 10205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:46.191593 10259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:46.192222 10263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:46.192298 10258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:46.192764 10205 server_base.cc:1034] running on GCE node
I20260811 00:22:46.193481 10205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:46.194541 10205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:46.195792 10205 hybrid_clock.cc:648] HybridClock initialized: now 1786407766195742 us; error 36 us; skew 500 ppm
I20260811 00:22:46.196206 10205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:46 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407766, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:46.199688 10205 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:22:46.201545 10205 webserver.cc:460] Webserver started at http://127.16.71.129:33947/ using document root <none> and password file <none>
I20260811 00:22:46.202320 10205 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:46.202467 10205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:46.209591 10205 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:22:46.212802 10283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:46.214455 10205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:22:46.214668 10205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "7a5da02d8eb39302cc5045a9aae61e8a"
server_key_iv: "04f0d18a0e885e548ff70e3a6556b769"
server_key_version: "encryptionkey@0"
I20260811 00:22:46.215974 10205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:46.254120 10205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:46.259272 10205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:46.259562 10205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:46.260710 10205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:46.262269 10205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:46.262367 10205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:46.262470 10205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:46.262547 10205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:46.298312 10205 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:45615
I20260811 00:22:46.298418 10484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:45615 every 8 connection(s)
I20260811 00:22:46.300092 10205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:22:46.310084 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 10205
I20260811 00:22:46.310904 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:38641
--local_ip_for_outbound_sockets=127.16.71.130
--tserver_master_addrs=127.16.71.190:36785
--webserver_port=45505
--webserver_interface=127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:22:46 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407766, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:46.328729 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:46.303748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:40157 (local address 127.16.71.190:36785)
0811 00:22:46.304550 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:46.304570 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:46.305273 (+ 703us) server_negotiation.cc:407] Connection header received
0811 00:22:46.307907 (+ 2634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:46.307922 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:46.308342 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:46.308832 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:46.311185 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.312545 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.314829 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.315540 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.315715 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:46.320496 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:46.320507 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:46.320525 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:46.320598 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:46.323685 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:46.324602 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:46.324642 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:46.324652 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:46.324850 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:46.325327 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:46.325331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:46.325333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:46.325877 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:46.326059 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:46.326920 (+ 861us) server_negotiation.cc:299] Negotiation successful
0811 00:22:46.327333 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":388,"threads_started":1}
I20260811 00:22:46.328713 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:46.304053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:40157)
0811 00:22:46.305053 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:46.305099 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:46.307039 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:46.309081 (+ 2042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:46.309093 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:46.309963 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:46.310814 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:46.310847 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.312788 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.312793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:46.314633 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:46.314651 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.315850 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.315857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:46.316153 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:46.317081 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:46.317113 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:46.320060 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:46.323864 (+ 3804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:46.323873 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:46.323887 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:46.324424 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:46.324987 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:46.324990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:46.324992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:46.325206 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:46.326041 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:46.326049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:46.326385 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 00:22:46.326655 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 00:22:46.327294 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":435,"threads_started":1}
I20260811 00:22:46.332154 10486 heartbeater.cc:347] Connected to a master server at 127.16.71.190:36785
I20260811 00:22:46.332470 10486 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:46.333299 10486 heartbeater.cc:511] Master 127.16.71.190:36785 requested a full tablet report, sending...
I20260811 00:22:46.335630 10136 ts_manager.cc:194] Registered new tserver with Master: 0896a57ff3a64cf4bd7ee2c07c7a9ab7 (127.16.71.129:45615)
I20260811 00:22:46.337195 10136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:40157
W20260811 00:22:46.675326 10492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:46.675684 10492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:46.675818 10492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:46.683564 10492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:46.683722 10492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:46.683879 10492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:22:46.694257 10492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:38641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=45505
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:46.695734 10492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:46.697263 10492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:46.711588 10505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:46.711616 10508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:46.712299 10503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:46.712767 10492 server_base.cc:1034] running on GCE node
I20260811 00:22:46.713618 10492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:46.714762 10492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:46.716053 10492 hybrid_clock.cc:648] HybridClock initialized: now 1786407766715986 us; error 50 us; skew 500 ppm
I20260811 00:22:46.716435 10492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:46 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407766, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:46.720480 10492 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:22:46.722450 10492 webserver.cc:460] Webserver started at http://127.16.71.130:45505/ using document root <none> and password file <none>
I20260811 00:22:46.723248 10492 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:46.723399 10492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:46.729663 10492 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:22:46.733214 10520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:46.735024 10492 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 00:22:46.735239 10492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "1db7f497d1514eb0ac71ac4a8b2d8289"
format_stamp: "Formatted at 2026-08-11 00:22:43 on dist-test-slave-f7th"
server_key: "17fd754990c94d2ccc4bf3554887f561"
server_key_iv: "37c0e9732198bb54daa5967de44db1a0"
server_key_version: "encryptionkey@0"
I20260811 00:22:46.736295 10492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:46.764487 10492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:46.769618 10492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:46.769927 10492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:46.771047 10492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:46.772542 10492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:46.772671 10492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:46.772782 10492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:46.772853 10492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:46.806715 10492 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:38641
I20260811 00:22:46.806794 10707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:38641 every 8 connection(s)
I20260811 00:22:46.808369 10492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:22:46.814167 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 10492
I20260811 00:22:46.814851 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:45973
--local_ip_for_outbound_sockets=127.16.71.131
--tserver_master_addrs=127.16.71.190:36785
--webserver_port=36147
--webserver_interface=127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:22:46 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407766, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:46.833947 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:46.811243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:35671 (local address 127.16.71.190:36785)
0811 00:22:46.811430 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:46.811436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:46.812655 (+ 1219us) server_negotiation.cc:407] Connection header received
0811 00:22:46.815009 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:46.815015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:46.815168 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:46.815267 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:46.817391 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.818149 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.819964 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.820511 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.820688 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:46.825772 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:46.825780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:46.825784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:46.825840 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:46.829444 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:46.830692 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:46.830699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:46.830702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:46.830902 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:46.831557 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:46.831563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:46.831566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:46.831960 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:46.832129 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:46.833282 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0811 00:22:46.833601 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:22:46.834923 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:46.811689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:35671)
0811 00:22:46.812455 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:46.812517 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:46.814639 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:46.815595 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:46.815605 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:46.816220 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:46.817139 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:46.817168 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.818350 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.818354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:46.819772 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:46.819789 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:46.820872 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:46.820880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:46.821239 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:46.822069 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:46.822099 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:46.825318 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:46.829762 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:46.829775 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:46.829799 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:46.830484 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:46.831080 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:46.831085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:46.831088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:46.831378 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:46.832194 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:46.832202 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:46.832755 (+ 553us) client_negotiation.cc:769] Sending connection context
0811 00:22:46.833081 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 00:22:46.833595 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":308,"threads_started":1}
I20260811 00:22:46.836887 10710 heartbeater.cc:347] Connected to a master server at 127.16.71.190:36785
I20260811 00:22:46.837334 10710 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:46.838490 10710 heartbeater.cc:511] Master 127.16.71.190:36785 requested a full tablet report, sending...
I20260811 00:22:46.840669 10136 ts_manager.cc:194] Registered new tserver with Master: 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641)
I20260811 00:22:46.841755 10136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:35671
W20260811 00:22:47.182616 10714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:47.182940 10714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:47.183073 10714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:47.191075 10714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:47.191223 10714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:47.191375 10714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:22:47.201936 10714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:45973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=36147
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:47.203428 10714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:47.204941 10714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:47.220877 10727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:47.220851 10720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:47.220948 10722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:47.227049 10714 server_base.cc:1034] running on GCE node
I20260811 00:22:47.227977 10714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:47.229257 10714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:47.230504 10714 hybrid_clock.cc:648] HybridClock initialized: now 1786407767230424 us; error 68 us; skew 500 ppm
I20260811 00:22:47.230875 10714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:47 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407767, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:47.234714 10714 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:22:47.236605 10714 webserver.cc:460] Webserver started at http://127.16.71.131:36147/ using document root <none> and password file <none>
I20260811 00:22:47.237504 10714 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:47.237666 10714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:47.244421 10714 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:22:47.250077 10742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:47.252640 10714 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.003s
I20260811 00:22:47.252923 10714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "753ddd2142b24d35bb02ea5ae47d2794"
format_stamp: "Formatted at 2026-08-11 00:22:44 on dist-test-slave-f7th"
server_key: "7c9afd3bd92a9c45ebb15c3a2765f8e0"
server_key_iv: "615bfc746f499b59d0bbba282f9ef831"
server_key_version: "encryptionkey@0"
I20260811 00:22:47.254468 10714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:47.271629 10714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:47.276245 10714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:47.276510 10714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:47.277603 10714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:47.279156 10714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:47.279258 10714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:47.279343 10714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:47.279407 10714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:47.313450 10714 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:45973
I20260811 00:22:47.313561 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:45973 every 8 connection(s)
I20260811 00:22:47.315073 10714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:22:47.321063 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 10714
Aug 11 00:22:47 dist-test-slave-f7th krb5kdc[9053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407767, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:22:47.337235 10977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.322037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34602 (local address 127.16.71.190:36785)
0811 00:22:47.322492 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.322498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.322850 (+ 352us) server_negotiation.cc:407] Connection header received
0811 00:22:47.323017 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.323021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.323081 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.323169 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:47.324932 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.325685 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.327113 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.327557 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.327667 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.330678 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:47.330684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:47.330687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:47.330727 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:47.334029 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:47.334849 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:47.334860 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:47.334862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:47.335018 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:47.335461 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:47.335466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:47.335469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:47.335905 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:47.336166 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.336508 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.336782 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":328,"thread_start_us":227,"threads_started":1}
I20260811 00:22:47.337478 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.317957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:45743 (local address 127.16.71.190:36785)
0811 00:22:47.318217 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.318224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.319496 (+ 1272us) server_negotiation.cc:407] Connection header received
0811 00:22:47.321633 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.321639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.321732 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.321838 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:47.323873 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.324689 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.326405 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.326841 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.326960 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.331038 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:47.331044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:47.331047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:47.331100 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:47.334068 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:47.334895 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:47.334900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:47.334902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:47.335031 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:47.335492 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:47.335496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:47.335498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:47.335881 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:47.336025 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.336855 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.337078 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":99}
I20260811 00:22:47.338698 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.318422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:45743)
0811 00:22:47.319308 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:47.319353 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:47.321307 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:47.322188 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:47.322200 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:47.322963 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:22:47.323672 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.323694 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.324872 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.324876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.326236 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.326251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.327099 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.327107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.327365 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.328081 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:22:47.328106 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:22:47.330785 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:22:47.334253 (+ 3468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:47.334262 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:47.334279 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:47.334763 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:47.335149 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:22:47.335155 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:22:47.335159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:22:47.335377 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:22:47.336030 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:22:47.336041 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:22:47.336425 (+ 384us) client_negotiation.cc:769] Sending connection context
0811 00:22:47.336760 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 00:22:47.337370 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":369,"threads_started":1}
I20260811 00:22:47.340163 10971 heartbeater.cc:347] Connected to a master server at 127.16.71.190:36785
I20260811 00:22:47.340523 10971 heartbeater.cc:464] Registering TS with master...
I20260811 00:22:47.340595 10486 heartbeater.cc:503] Master 127.16.71.190:36785 was elected leader, sending a full tablet report...
I20260811 00:22:47.341540 10971 heartbeater.cc:511] Master 127.16.71.190:36785 requested a full tablet report, sending...
I20260811 00:22:47.343116 10136 ts_manager.cc:194] Registered new tserver with Master: 753ddd2142b24d35bb02ea5ae47d2794 (127.16.71.131:45973)
I20260811 00:22:47.343909 10136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:45743
I20260811 00:22:47.352820 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:22:47.360968 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.353841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.16.71.190:36785)
0811 00:22:47.354032 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.354039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.354484 (+ 445us) server_negotiation.cc:407] Connection header received
0811 00:22:47.354764 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.354769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.354989 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.355161 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:22:47.355658 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.356477 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.358551 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.359050 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.359155 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.359788 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:22:47.360418 (+ 630us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:22:47.360539 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.360691 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.360770 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:22:47.362128 10136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:22:47.369678 10136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:22:47.404534 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.383373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.16.71.130:38641)
0811 00:22:47.394080 (+ 10707us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.394091 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.395185 (+ 1094us) server_negotiation.cc:407] Connection header received
0811 00:22:47.395298 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.395307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.395598 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.397130 (+ 1532us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:47.397408 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.399195 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.401836 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.403237 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.403381 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.403610 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.404024 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.404202 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10492,"thread_start_us":10337,"threads_started":1}
I20260811 00:22:47.404660 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.383282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:38641 (local address 127.0.0.1:52394)
0811 00:22:47.394660 (+ 11378us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:47.394716 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:47.394955 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:47.396200 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:47.396209 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:47.396247 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:47.396865 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.396881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.399390 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.399396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.401654 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.401669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.403564 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.403571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.403748 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.403812 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 00:22:47.403930 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 00:22:47.404130 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11079,"mutex_wait_us":2,"thread_start_us":10896,"threads_started":1}
I20260811 00:22:47.406585 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.383724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:45615 (local address 127.0.0.1:48772)
0811 00:22:47.397865 (+ 14141us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:47.397883 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:47.398158 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:47.399049 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:47.399054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:47.399077 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:47.399566 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.399578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.400989 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.400994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.403815 (+ 2821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.403837 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.405751 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.405758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.405925 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.405960 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:22:47.406047 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:22:47.406178 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14006,"mutex_wait_us":102,"thread_start_us":13860,"threads_started":1}
I20260811 00:22:47.406585 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.384363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.16.71.129:45615)
0811 00:22:47.397383 (+ 13020us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.397393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.398047 (+ 654us) server_negotiation.cc:407] Connection header received
0811 00:22:47.398387 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.398396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.398623 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.398791 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:47.399766 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.400841 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.404023 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.405513 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.405653 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.405886 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.406113 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.406289 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12827,"thread_start_us":12669,"threads_started":1}
I20260811 00:22:47.406211 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.383483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:45973 (local address 127.0.0.1:44864)
0811 00:22:47.396954 (+ 13471us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:47.396973 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:47.397096 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:47.397907 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:47.397912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:47.397944 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:47.398339 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.398352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.400979 (+ 2627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.400985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.403301 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.403328 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.405321 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.405328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.405494 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.405527 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:22:47.405620 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:22:47.405853 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13317,"thread_start_us":13128,"threads_started":1}
I20260811 00:22:47.407299 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.384112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44864 (local address 127.16.71.131:45973)
0811 00:22:47.396397 (+ 12285us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.396407 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.397234 (+ 827us) server_negotiation.cc:407] Connection header received
0811 00:22:47.397343 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.397350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.397623 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.397831 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:47.398795 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.400814 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.403516 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.404999 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.405132 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.405372 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.406326 (+ 954us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.406522 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12089,"thread_start_us":11928,"threads_started":1}
I20260811 00:22:47.409436 10600 tablet_service.cc:1467] Processing CreateTablet for tablet 80769a399c1240d1a24d6212bd5a6c80 (DEFAULT_TABLE table=test-table [id=bb5bf196f435440fa26a2a8249a736f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:47.410737 10600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80769a399c1240d1a24d6212bd5a6c80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:47.411062 10843 tablet_service.cc:1467] Processing CreateTablet for tablet 80769a399c1240d1a24d6212bd5a6c80 (DEFAULT_TABLE table=test-table [id=bb5bf196f435440fa26a2a8249a736f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:47.411062 10378 tablet_service.cc:1467] Processing CreateTablet for tablet 80769a399c1240d1a24d6212bd5a6c80 (DEFAULT_TABLE table=test-table [id=bb5bf196f435440fa26a2a8249a736f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:22:47.412566 10843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80769a399c1240d1a24d6212bd5a6c80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:47.412592 10378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80769a399c1240d1a24d6212bd5a6c80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:22:47.425186 11002 tablet_bootstrap.cc:492] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Bootstrap starting.
I20260811 00:22:47.429431 11005 tablet_bootstrap.cc:492] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Bootstrap starting.
I20260811 00:22:47.430190 11002 tablet_bootstrap.cc:654] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:47.430511 11006 tablet_bootstrap.cc:492] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Bootstrap starting.
I20260811 00:22:47.431687 11002 log.cc:824] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:47.434039 11005 tablet_bootstrap.cc:654] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:47.434152 11006 tablet_bootstrap.cc:654] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Neither blocks nor log segments found. Creating new log.
I20260811 00:22:47.435154 11005 log.cc:824] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:47.435459 11006 log.cc:824] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:47.435552 11002 tablet_bootstrap.cc:492] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: No bootstrap required, opened a new log
I20260811 00:22:47.435853 11002 ts_tablet_manager.cc:1397] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 00:22:47.439005 11005 tablet_bootstrap.cc:492] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: No bootstrap required, opened a new log
I20260811 00:22:47.439045 11006 tablet_bootstrap.cc:492] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: No bootstrap required, opened a new log
I20260811 00:22:47.439289 11005 ts_tablet_manager.cc:1397] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260811 00:22:47.439307 11006 ts_tablet_manager.cc:1397] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:22:47.447798 11005 raft_consensus.cc:348] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.447926 11002 raft_consensus.cc:348] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.448414 11002 raft_consensus.cc:374] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:47.448418 11005 raft_consensus.cc:374] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:47.448537 11002 raft_consensus.cc:729] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1db7f497d1514eb0ac71ac4a8b2d8289, State: Initialized, Role: FOLLOWER
I20260811 00:22:47.448653 11005 raft_consensus.cc:729] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0896a57ff3a64cf4bd7ee2c07c7a9ab7, State: Initialized, Role: FOLLOWER
I20260811 00:22:47.449368 11005 consensus_queue.cc:260] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.449326 11002 consensus_queue.cc:260] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.450584 11006 raft_consensus.cc:348] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.451098 11006 raft_consensus.cc:374] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:22:47.451154 10710 heartbeater.cc:503] Master 127.16.71.190:36785 was elected leader, sending a full tablet report...
I20260811 00:22:47.451202 11006 raft_consensus.cc:729] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753ddd2142b24d35bb02ea5ae47d2794, State: Initialized, Role: FOLLOWER
I20260811 00:22:47.451565 11005 ts_tablet_manager.cc:1428] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260811 00:22:47.451932 11006 consensus_queue.cc:260] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.453320 11002 ts_tablet_manager.cc:1428] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260811 00:22:47.454474 10971 heartbeater.cc:503] Master 127.16.71.190:36785 was elected leader, sending a full tablet report...
I20260811 00:22:47.454496 11006 ts_tablet_manager.cc:1428] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
W20260811 00:22:47.552845 10487 tablet.cc:2369] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:22:47.560544 10711 tablet.cc:2369] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:22:47.566756 10972 tablet.cc:2369] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:22:47.730515 11018 raft_consensus.cc:482] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:22:47.730887 11018 raft_consensus.cc:504] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.732838 11018 leader_election.cc:288] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641), 0896a57ff3a64cf4bd7ee2c07c7a9ab7 (127.16.71.129:45615)
I20260811 00:22:47.738934 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.733347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:37733 (local address 127.16.71.129:45615)
0811 00:22:47.733564 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.733570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.733690 (+ 120us) server_negotiation.cc:407] Connection header received
0811 00:22:47.733906 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.733910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.733977 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.734065 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:47.734489 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.735243 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.737184 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.738009 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.738180 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.738261 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.738555 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.738689 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:22:47.738947 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.733373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:45615 (local address 127.16.71.131:37733)
0811 00:22:47.733591 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:47.733613 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:47.733766 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:47.734103 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:47.734112 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:47.734139 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:47.734346 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.734359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.735409 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.735413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.737022 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.737039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.738180 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.738186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.738394 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.738420 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 00:22:47.738506 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 00:22:47.738654 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":127}
I20260811 00:22:47.739746 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.733370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:43225 (local address 127.16.71.130:38641)
0811 00:22:47.733589 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.733595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.734601 (+ 1006us) server_negotiation.cc:407] Connection header received
0811 00:22:47.734757 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.734761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.734824 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.734911 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:22:47.735439 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.736129 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.738060 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.738845 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.739004 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.739081 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.739357 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.739495 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:22:47.739861 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.733758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:38641 (local address 127.16.71.131:43225)
0811 00:22:47.734479 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:47.734499 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:22:47.734627 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:22:47.734947 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:22:47.734951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:22:47.734977 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:22:47.735279 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.735289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.736285 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.736289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.737882 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:22:47.737897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.739053 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.739059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:22:47.739207 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.739231 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:22:47.739310 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 00:22:47.739442 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":61,"thread_start_us":409,"threads_started":1}
I20260811 00:22:47.740037 10411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80769a399c1240d1a24d6212bd5a6c80" candidate_uuid: "753ddd2142b24d35bb02ea5ae47d2794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" is_pre_election: true
I20260811 00:22:47.740765 10411 raft_consensus.cc:2454] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753ddd2142b24d35bb02ea5ae47d2794 in term 0.
I20260811 00:22:47.740883 10631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80769a399c1240d1a24d6212bd5a6c80" candidate_uuid: "753ddd2142b24d35bb02ea5ae47d2794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" is_pre_election: true
I20260811 00:22:47.741456 10631 raft_consensus.cc:2454] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753ddd2142b24d35bb02ea5ae47d2794 in term 0.
I20260811 00:22:47.741614 10752 leader_election.cc:302] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0896a57ff3a64cf4bd7ee2c07c7a9ab7, 753ddd2142b24d35bb02ea5ae47d2794; no voters:
I20260811 00:22:47.742040 11018 raft_consensus.cc:2781] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:22:47.742213 11018 raft_consensus.cc:482] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:22:47.742328 11018 raft_consensus.cc:3037] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:47.744804 11018 raft_consensus.cc:504] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.745540 11018 leader_election.cc:288] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [CANDIDATE]: Term 1 election: Requested vote from peers 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641), 0896a57ff3a64cf4bd7ee2c07c7a9ab7 (127.16.71.129:45615)
I20260811 00:22:47.746090 10631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80769a399c1240d1a24d6212bd5a6c80" candidate_uuid: "753ddd2142b24d35bb02ea5ae47d2794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289"
I20260811 00:22:47.746181 10411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80769a399c1240d1a24d6212bd5a6c80" candidate_uuid: "753ddd2142b24d35bb02ea5ae47d2794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7"
I20260811 00:22:47.746345 10631 raft_consensus.cc:3037] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:47.746402 10411 raft_consensus.cc:3037] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:22:47.749819 10411 raft_consensus.cc:2454] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753ddd2142b24d35bb02ea5ae47d2794 in term 1.
I20260811 00:22:47.749819 10631 raft_consensus.cc:2454] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753ddd2142b24d35bb02ea5ae47d2794 in term 1.
I20260811 00:22:47.750492 10749 leader_election.cc:302] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1db7f497d1514eb0ac71ac4a8b2d8289, 753ddd2142b24d35bb02ea5ae47d2794; no voters:
I20260811 00:22:47.750874 11018 raft_consensus.cc:2781] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:22:47.751334 11018 raft_consensus.cc:686] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 LEADER]: Becoming Leader. State: Replica: 753ddd2142b24d35bb02ea5ae47d2794, State: Running, Role: LEADER
I20260811 00:22:47.751853 11018 consensus_queue.cc:237] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } }
I20260811 00:22:47.756757 10133 catalog_manager.cc:5495] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753ddd2142b24d35bb02ea5ae47d2794 (127.16.71.131). New cstate: current_term: 1 leader_uuid: "753ddd2142b24d35bb02ea5ae47d2794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753ddd2142b24d35bb02ea5ae47d2794" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 } health_report { overall_health: UNKNOWN } } }
I20260811 00:22:47.783262 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.777084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.16.71.131:45973)
0811 00:22:47.777386 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.777392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.777426 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:22:47.777661 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.777665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.777751 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.777920 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:22:47.778290 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.779042 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.780753 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.781200 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.781316 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.781974 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:22:47.782558 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:22:47.782686 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.782868 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.782965 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 00:22:47.793155 10411 raft_consensus.cc:1261] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 FOLLOWER]: Refusing update from remote peer 753ddd2142b24d35bb02ea5ae47d2794: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:47.793155 10631 raft_consensus.cc:1261] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Refusing update from remote peer 753ddd2142b24d35bb02ea5ae47d2794: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:22:47.794507 11020 consensus_queue.cc:1035] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 45615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:22:47.794945 11018 consensus_queue.cc:1035] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1db7f497d1514eb0ac71ac4a8b2d8289" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:22:47.810743 11024 mvcc.cc:205] Tried to move back new op lower bound from 7317126216867086336 to 7317126216716148736. Current Snapshot: MvccSnapshot[applied={T|T < 7317126216867086336}]
I20260811 00:22:47.834034 10133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:22:47.834384 10133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:22:47.839170 10133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Sending DeleteTablet for 3 replicas of tablet 80769a399c1240d1a24d6212bd5a6c80
I20260811 00:22:47.840553 10843 tablet_service.cc:1514] Processing DeleteTablet for tablet 80769a399c1240d1a24d6212bd5a6c80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:47 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:44864
I20260811 00:22:47.841785 10378 tablet_service.cc:1514] Processing DeleteTablet for tablet 80769a399c1240d1a24d6212bd5a6c80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:47 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:48772
I20260811 00:22:47.841845 10600 tablet_service.cc:1514] Processing DeleteTablet for tablet 80769a399c1240d1a24d6212bd5a6c80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:22:47 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:52394
I20260811 00:22:47.842983 11049 tablet_replica.cc:330] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: stopping tablet replica
I20260811 00:22:47.843362 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 10205
I20260811 00:22:47.843744 11049 raft_consensus.cc:2229] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:22:47.844931 11049 raft_consensus.cc:2258] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:47.847594 11049 ts_tablet_manager.cc:1905] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:47.851600 11050 tablet_replica.cc:330] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: stopping tablet replica
I20260811 00:22:47.852689 11050 raft_consensus.cc:2229] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:22:47.853304 11050 raft_consensus.cc:2258] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:47.856295 11049 ts_tablet_manager.cc:1918] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:47.856602 11049 log.cc:1196] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/80769a399c1240d1a24d6212bd5a6c80
I20260811 00:22:47.857369 11049 ts_tablet_manager.cc:1939] T 80769a399c1240d1a24d6212bd5a6c80 P 0896a57ff3a64cf4bd7ee2c07c7a9ab7: Deleting consensus metadata
I20260811 00:22:47.859859 11050 ts_tablet_manager.cc:1905] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:47.859901 10111 catalog_manager.cc:4841] TS 0896a57ff3a64cf4bd7ee2c07c7a9ab7 (127.16.71.129:45615): tablet 80769a399c1240d1a24d6212bd5a6c80 (table test-table [id=bb5bf196f435440fa26a2a8249a736f3]) successfully deleted
I20260811 00:22:47.865294 11048 tablet_replica.cc:330] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: stopping tablet replica
I20260811 00:22:47.865854 11048 raft_consensus.cc:2229] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:22:47.868000 11050 ts_tablet_manager.cc:1918] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:47.868209 11050 log.cc:1196] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/80769a399c1240d1a24d6212bd5a6c80
I20260811 00:22:47.868877 11050 ts_tablet_manager.cc:1939] T 80769a399c1240d1a24d6212bd5a6c80 P 1db7f497d1514eb0ac71ac4a8b2d8289: Deleting consensus metadata
I20260811 00:22:47.870208 11048 raft_consensus.cc:2258] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:22:47.870966 10110 catalog_manager.cc:4841] TS 1db7f497d1514eb0ac71ac4a8b2d8289 (127.16.71.130:38641): tablet 80769a399c1240d1a24d6212bd5a6c80 (table test-table [id=bb5bf196f435440fa26a2a8249a736f3]) successfully deleted
I20260811 00:22:47.872848 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:47.844295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.16.71.129:45615)
0811 00:22:47.844487 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:47.844492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:47.855167 (+ 10675us) server_negotiation.cc:407] Connection header received
0811 00:22:47.855450 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:47.855456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:47.855547 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:47.855615 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:47.858526 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.859637 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.861450 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:47.861875 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:47.861955 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:47.866050 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:47.866067 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:47.866083 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:47.866124 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:47.869423 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:47.870398 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:47.870407 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:47.870409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:47.870559 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:47.871188 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:47.871194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:47.871198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:47.871590 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:47.871892 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:47.872245 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 00:22:47.872472 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 00:22:47.873481 10477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:47.874327 11048 ts_tablet_manager.cc:1905] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:22:47.883061 11048 ts_tablet_manager.cc:1918] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:22:47.883412 11048 log.cc:1196] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/80769a399c1240d1a24d6212bd5a6c80
I20260811 00:22:47.884109 11048 ts_tablet_manager.cc:1939] T 80769a399c1240d1a24d6212bd5a6c80 P 753ddd2142b24d35bb02ea5ae47d2794: Deleting consensus metadata
I20260811 00:22:47.886372 10112 catalog_manager.cc:4841] TS 753ddd2142b24d35bb02ea5ae47d2794 (127.16.71.131:45973): tablet 80769a399c1240d1a24d6212bd5a6c80 (table test-table [id=bb5bf196f435440fa26a2a8249a736f3]) successfully deleted
I20260811 00:22:47.967702 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 10205
I20260811 00:22:48.003042 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 10492
I20260811 00:22:48.024174 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:48.003941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.16.71.130:38641)
0811 00:22:48.004183 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:48.004190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:48.004215 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:22:48.004375 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:48.004382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:48.004476 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:48.004594 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:48.006160 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:48.007347 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:48.008893 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:48.009562 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:48.009709 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:48.016160 (+ 6451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:48.016183 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:48.016206 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:48.016265 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:48.020296 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:48.021534 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:48.021546 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:48.021550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:48.021753 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:48.022233 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:48.022239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:48.022243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:48.022817 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:48.023154 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:48.023506 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 00:22:48.023782 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:22:48.025076 10702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:48.116950 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 10492
I20260811 00:22:48.147255 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 10714
I20260811 00:22:48.165292 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:48.148214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.16.71.131:45973)
0811 00:22:48.148379 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 00:22:48.148384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:22:48.148404 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:22:48.148597 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:22:48.148601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:22:48.148849 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:22:48.148938 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:22:48.150993 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:48.151943 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:48.154064 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:22:48.154540 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:22:48.154659 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:22:48.158365 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:22:48.158389 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:22:48.158416 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:22:48.158482 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:22:48.161948 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:48.162949 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:48.162961 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:48.162964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:48.163139 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:22:48.163628 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:22:48.163633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:22:48.163635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:22:48.164082 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:22:48.164354 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 00:22:48.164774 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 00:22:48.164985 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:22:48.165969 10961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:48.292122 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 10714
I20260811 00:22:48.316457 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 10064
I20260811 00:22:48.317689 10195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:22:48.434935 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 10064
I20260811 00:22:48.466876 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36785
--webserver_interface=127.16.71.190
--webserver_port=35755
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:36785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:22:48.831341 11062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:48.831691 11062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:48.831830 11062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:48.839468 11062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:22:48.839627 11062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:48.839700 11062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:48.839782 11062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:22:48.839843 11062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:22:48.850436 11062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:36785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=35755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:48.851997 11062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:48.853540 11062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:48.867398 11081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:48.867443 11075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:48.867795 11073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:48.868541 11062 server_base.cc:1034] running on GCE node
I20260811 00:22:48.869511 11062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:48.871256 11062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:48.872545 11062 hybrid_clock.cc:648] HybridClock initialized: now 1786407768872464 us; error 47 us; skew 500 ppm
I20260811 00:22:48.872992 11062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:48 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407768, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:48.876653 11062 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:22:48.878489 11062 webserver.cc:460] Webserver started at http://127.16.71.190:35755/ using document root <none> and password file <none>
I20260811 00:22:48.879310 11062 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:48.879457 11062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:48.886264 11062 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:22:48.889757 11091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:48.891546 11062 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:22:48.891794 11062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "a6885e0ac8ef45bb91f8413d679637a2"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "0a1862cb07ca4d04f9b90c02ed9d3d39"
server_key_iv: "715b4bfadb84825e1e056b936c1ec38e"
server_key_version: "encryptionkey@0"
I20260811 00:22:48.893042 11062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:48.906068 11062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:48.910939 11062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:48.911264 11062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:48.932274 11062 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:36785
I20260811 00:22:48.932286 11192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:36785 every 8 connection(s)
I20260811 00:22:48.933979 11062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:22:48.939554 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 11062
I20260811 00:22:48.940662 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:45615
--local_ip_for_outbound_sockets=127.16.71.129
--tserver_master_addrs=127.16.71.190:36785
--webserver_port=33947
--webserver_interface=127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:22:48.943058 11193 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:22:48.948113 11193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Bootstrap starting.
I20260811 00:22:48.971625 11193 log.cc:824] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:22:48.991043 11193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:22:48.991896 11193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2: Bootstrap complete.
I20260811 00:22:49.003566 11193 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:49.004971 11193 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6885e0ac8ef45bb91f8413d679637a2, State: Initialized, Role: FOLLOWER
I20260811 00:22:49.005832 11193 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:49.006233 11193 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:22:49.006389 11193 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:22:49.006590 11193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:22:49.011229 11193 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:49.011901 11193 leader_election.cc:302] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6885e0ac8ef45bb91f8413d679637a2; no voters:
I20260811 00:22:49.012686 11193 leader_election.cc:288] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 00:22:49.012861 11198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:22:49.014258 11198 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [term 3 LEADER]: Becoming Leader. State: Replica: a6885e0ac8ef45bb91f8413d679637a2, State: Running, Role: LEADER
I20260811 00:22:49.014882 11198 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } }
I20260811 00:22:49.015466 11193 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:22:49.018693 11203 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6885e0ac8ef45bb91f8413d679637a2. Latest consensus state: current_term: 3 leader_uuid: "a6885e0ac8ef45bb91f8413d679637a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } } }
I20260811 00:22:49.019385 11203 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:49.018913 11200 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a6885e0ac8ef45bb91f8413d679637a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6885e0ac8ef45bb91f8413d679637a2" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36785 } } }
I20260811 00:22:49.020901 11200 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6885e0ac8ef45bb91f8413d679637a2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:22:49.031031 11226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:22:49.041235 11226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:22:49.042728 11226 catalog_manager.cc:1259] Loaded cluster ID: e778653fa6594d8fa2b1a9b921fe9efb
I20260811 00:22:49.042827 11226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 00:22:49.044380 11226 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc467e479ad google::LogMessage::Fail() at ??:0
@ 0x7fc467e4a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc467e474de google::LogMessage::Flush() at ??:0
@ 0x7fc467e48a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc47d87e512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc47d87b976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc47d903610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
I20260811 00:22:49.077534 11241 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fc47d903439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc46aa93345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc46aa9691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc46aa96799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc46aa71eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc46dd4b6db start_thread at ??:0
@ 0x7fc466ce161f clone at ??:0
W20260811 00:22:49.411520 11196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:49.411845 11196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:49.411945 11196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:49.419615 11196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:49.419755 11196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:49.419873 11196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:22:49.430362 11196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:45615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=33947
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:49.431782 11196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:49.433238 11196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:49.449179 11277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:49.449139 11275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:49.449203 11272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:49.450289 11196 server_base.cc:1034] running on GCE node
I20260811 00:22:49.451071 11196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:49.452242 11196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:49.453504 11196 hybrid_clock.cc:648] HybridClock initialized: now 1786407769453416 us; error 80 us; skew 500 ppm
I20260811 00:22:49.453922 11196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:49 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407769, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:49.458024 11196 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:22:49.460053 11196 webserver.cc:460] Webserver started at http://127.16.71.129:33947/ using document root <none> and password file <none>
I20260811 00:22:49.460997 11196 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:49.461160 11196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:49.467983 11196 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:22:49.471643 11293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:49.473582 11196 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 00:22:49.473805 11196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "0896a57ff3a64cf4bd7ee2c07c7a9ab7"
format_stamp: "Formatted at 2026-08-11 00:22:42 on dist-test-slave-f7th"
server_key: "7a5da02d8eb39302cc5045a9aae61e8a"
server_key_iv: "04f0d18a0e885e548ff70e3a6556b769"
server_key_version: "encryptionkey@0"
I20260811 00:22:49.474998 11196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:49.501449 11196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:49.506335 11196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:49.506660 11196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:49.507804 11196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:49.509325 11196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:49.509435 11196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:49.509560 11196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:49.509636 11196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:49.545226 11196 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:45615
I20260811 00:22:49.545313 11516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:45615 every 8 connection(s)
I20260811 00:22:49.546948 11196 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:22:49.547434 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 11196
I20260811 00:22:49.548197 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:38641
--local_ip_for_outbound_sockets=127.16.71.130
--tserver_master_addrs=127.16.71.190:36785
--webserver_port=45505
--webserver_interface=127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:22:49.552884 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:49.550552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:35569)
0811 00:22:49.551433 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:49.551686 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":370,"threads_started":1}
W20260811 00:22:49.553706 11517 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:36785 (0 consecutive failures): Network error: Failed to ping master at 127.16.71.190:36785: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
I20260811 00:22:49.554627 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:49.554271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:52819)
0811 00:22:49.554435 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:49.554512 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 00:22:49.555578 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:49.555243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:51047)
0811 00:22:49.555378 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:49.555460 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:22:49.555814 11517 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:22:49.914523 11521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:49.914883 11521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:49.915025 11521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:49.922904 11521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:49.923084 11521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:49.923250 11521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:22:49.934077 11521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:38641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=45505
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:49.935555 11521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:49.937320 11521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:49.952593 11536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:49.952812 11531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:49.952858 11532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:49.953680 11521 server_base.cc:1034] running on GCE node
I20260811 00:22:49.954484 11521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:49.955636 11521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:49.956871 11521 hybrid_clock.cc:648] HybridClock initialized: now 1786407769956782 us; error 83 us; skew 500 ppm
I20260811 00:22:49.957254 11521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:49 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407769, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:49.961190 11521 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:22:49.963102 11521 webserver.cc:460] Webserver started at http://127.16.71.130:45505/ using document root <none> and password file <none>
I20260811 00:22:49.963932 11521 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:49.964082 11521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:49.970448 11521 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 00:22:49.973655 11549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:49.975163 11521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:22:49.975348 11521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "1db7f497d1514eb0ac71ac4a8b2d8289"
format_stamp: "Formatted at 2026-08-11 00:22:43 on dist-test-slave-f7th"
server_key: "17fd754990c94d2ccc4bf3554887f561"
server_key_iv: "37c0e9732198bb54daa5967de44db1a0"
server_key_version: "encryptionkey@0"
I20260811 00:22:49.976429 11521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:49.991864 11521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:49.997098 11521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:49.997422 11521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:49.998543 11521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:50.000190 11521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:50.000311 11521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:50.000442 11521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:50.000519 11521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:50.034912 11521 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:38641
I20260811 00:22:50.034981 11763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:38641 every 8 connection(s)
I20260811 00:22:50.036602 11521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:22:50.041114 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 11521
I20260811 00:22:50.042066 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:45973
--local_ip_for_outbound_sockets=127.16.71.131
--tserver_master_addrs=127.16.71.190:36785
--webserver_port=36147
--webserver_interface=127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:22:50.042886 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:50.040047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:38069)
0811 00:22:50.041065 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:50.041425 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":373,"threads_started":1}
W20260811 00:22:50.043855 11764 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:36785 (0 consecutive failures): Network error: Failed to ping master at 127.16.71.190:36785: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
I20260811 00:22:50.044973 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:50.044496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:43657)
0811 00:22:50.044725 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:50.044822 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 00:22:50.046043 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:50.045624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:58463)
0811 00:22:50.045796 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:50.045896 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:22:50.046375 11764 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:22:50.395260 11770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:22:50.395574 11770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:22:50.395701 11770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:22:50.403241 11770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:22:50.403385 11770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:22:50.403541 11770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:22:50.413883 11770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:45973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=36147
--tserver_master_addrs=127.16.71.190:36785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:22:50.415328 11770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:22:50.416828 11770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:22:50.431214 11782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:50.431150 11784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:22:50.431186 11787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:22:50.432039 11770 server_base.cc:1034] running on GCE node
I20260811 00:22:50.432829 11770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:22:50.433903 11770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:22:50.435150 11770 hybrid_clock.cc:648] HybridClock initialized: now 1786407770435082 us; error 57 us; skew 500 ppm
I20260811 00:22:50.435521 11770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:22:50 dist-test-slave-f7th krb5kdc[9053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407770, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:22:50.439208 11770 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:22:50.441031 11770 webserver.cc:460] Webserver started at http://127.16.71.131:36147/ using document root <none> and password file <none>
I20260811 00:22:50.441915 11770 fs_manager.cc:359] Metadata directory not provided
I20260811 00:22:50.442066 11770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:22:50.448500 11770 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 00:22:50.452041 11800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:50.453855 11770 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 00:22:50.454077 11770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "753ddd2142b24d35bb02ea5ae47d2794"
format_stamp: "Formatted at 2026-08-11 00:22:44 on dist-test-slave-f7th"
server_key: "7c9afd3bd92a9c45ebb15c3a2765f8e0"
server_key_iv: "615bfc746f499b59d0bbba282f9ef831"
server_key_version: "encryptionkey@0"
I20260811 00:22:50.455312 11770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:22:50.476243 11770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:22:50.481637 11770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:22:50.481969 11770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:22:50.483103 11770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:22:50.484658 11770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:22:50.484786 11770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:50.484912 11770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:22:50.484988 11770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:22:50.518719 11770 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:45973
I20260811 00:22:50.518801 12015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:45973 every 8 connection(s)
I20260811 00:22:50.520308 11770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:22:50.523864 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 11770
I20260811 00:22:50.526573 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:50.523847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:37349)
0811 00:22:50.525001 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:50.525310 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":428,"threads_started":1}
W20260811 00:22:50.527496 12018 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:36785 (0 consecutive failures): Network error: Failed to ping master at 127.16.71.190:36785: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
I20260811 00:22:50.528461 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:50.528067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:36993)
0811 00:22:50.528242 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:50.528329 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 00:22:50.529467 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:50.529087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:45681)
0811 00:22:50.529242 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:50.529337 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 00:22:50.529721 12018 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:22:50.557631 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:50.556731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:51955)
0811 00:22:50.557250 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:50.557375 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":243,"threads_started":1}
I20260811 00:22:51.048103 12028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:51.047166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:60109)
0811 00:22:51.047703 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:51.047847 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":261,"threads_started":1}
I20260811 00:22:51.531590 12029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:51.530450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:35101)
0811 00:22:51.531135 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:51.531316 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":279,"threads_started":1}
I20260811 00:22:51.559587 12031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:51.558731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:40953)
0811 00:22:51.559273 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:51.559370 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":272,"threads_started":1}
I20260811 00:22:52.049921 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:52.049140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:33759)
0811 00:22:52.049651 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:52.049741 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":232,"threads_started":1}
I20260811 00:22:52.533634 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:52.532757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:38363)
0811 00:22:52.533337 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:52.533441 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":304,"threads_started":1}
I20260811 00:22:52.561700 12039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:52.560791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:51515)
0811 00:22:52.561399 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:52.561496 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":333,"threads_started":1}
I20260811 00:22:53.051831 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:53.051020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:49923)
0811 00:22:53.051528 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:53.051643 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":254,"threads_started":1}
I20260811 00:22:53.535494 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:53.534685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:39087)
0811 00:22:53.535221 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:53.535309 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":300,"threads_started":1}
I20260811 00:22:53.563567 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:53.562733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:46443)
0811 00:22:53.563289 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:53.563380 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":299,"threads_started":1}
I20260811 00:22:54.053824 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:54.053012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:43577)
0811 00:22:54.053523 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:54.053618 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":241,"threads_started":1}
I20260811 00:22:54.537470 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:54.536648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:50815)
0811 00:22:54.537179 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:54.537279 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":247,"threads_started":1}
I20260811 00:22:54.565339 12045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:54.564533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:57257)
0811 00:22:54.565072 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:54.565160 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
I20260811 00:22:55.055913 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:55.055093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:45365)
0811 00:22:55.055620 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:55.055720 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":235,"threads_started":1}
I20260811 00:22:55.539506 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:55.538649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:54513)
0811 00:22:55.539222 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:55.539316 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":289,"threads_started":1}
I20260811 00:22:55.567150 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:55.566277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:38597)
0811 00:22:55.566801 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:55.566925 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":262,"threads_started":1}
I20260811 00:22:56.057760 12049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:56.056946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:56845)
0811 00:22:56.057460 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:56.057577 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":285,"threads_started":1}
I20260811 00:22:56.541538 12053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:56.540662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:50951)
0811 00:22:56.541187 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:56.541291 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":254,"threads_started":1}
I20260811 00:22:56.569087 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:56.568216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:47945)
0811 00:22:56.568765 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:56.568855 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":215,"threads_started":1}
I20260811 00:22:57.060001 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:57.058922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:48667)
0811 00:22:57.059593 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:57.059714 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":299,"threads_started":1}
I20260811 00:22:57.543705 12060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:57.542765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:58915)
0811 00:22:57.543374 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:57.543472 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":279,"threads_started":1}
I20260811 00:22:57.571014 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:57.570200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:52167)
0811 00:22:57.570704 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:57.570806 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
I20260811 00:22:58.062328 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:58.061346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:48157)
0811 00:22:58.061986 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:58.062121 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":310,"threads_started":1}
I20260811 00:22:58.545780 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:58.544889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:32799)
0811 00:22:58.545453 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:58.545602 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":280,"threads_started":1}
I20260811 00:22:58.572952 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:58.572022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:41319)
0811 00:22:58.572565 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:58.572698 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":265,"threads_started":1}
I20260811 00:22:59.064237 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:59.063296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:43201)
0811 00:22:59.063846 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:59.063990 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":261,"threads_started":1}
I20260811 00:22:59.547881 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:59.546987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:37767)
0811 00:22:59.547600 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:59.547700 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":291,"threads_started":1}
I20260811 00:22:59.574890 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:22:59.574083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:46371)
0811 00:22:59.574616 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 00:22:59.574711 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":286,"threads_started":1}
I20260811 00:23:00.066238 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:00.065376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:56947)
0811 00:23:00.065948 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:00.066059 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":310,"threads_started":1}
I20260811 00:23:00.549612 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:00.548845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:56889)
0811 00:23:00.549354 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:00.549441 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":287,"threads_started":1}
I20260811 00:23:00.576994 12077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:00.576040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:60795)
0811 00:23:00.576659 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:00.576789 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":274,"threads_started":1}
I20260811 00:23:01.068305 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:01.067438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:49829)
0811 00:23:01.067996 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:01.068113 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":318,"threads_started":1}
I20260811 00:23:01.551445 12080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:01.550591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:57701)
0811 00:23:01.551122 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:01.551228 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":273,"threads_started":1}
I20260811 00:23:01.578732 12081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:01.577936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:45695)
0811 00:23:01.578428 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:01.578515 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":260,"threads_started":1}
I20260811 00:23:02.070429 12082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:02.069520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:52211)
0811 00:23:02.070154 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:02.070247 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":316,"threads_started":1}
I20260811 00:23:02.553242 12083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:02.552472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:40177)
0811 00:23:02.552977 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:02.553077 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":263,"threads_started":1}
I20260811 00:23:02.580873 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:02.579787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:57133)
0811 00:23:02.580505 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:02.580641 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":424,"threads_started":1}
I20260811 00:23:03.072773 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:03.071683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:44647)
0811 00:23:03.072307 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:03.072451 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":278,"threads_started":1}
I20260811 00:23:03.555186 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:03.554271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:54621)
0811 00:23:03.554875 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:03.554982 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":311,"threads_started":1}
I20260811 00:23:03.582713 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:03.581880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:55505)
0811 00:23:03.582391 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:03.582491 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
I20260811 00:23:04.074880 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:04.074063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:43275)
0811 00:23:04.074574 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:04.074682 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":239,"threads_started":1}
I20260811 00:23:04.557004 12093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:04.556171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:42931)
0811 00:23:04.556712 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:04.556805 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":256,"threads_started":1}
I20260811 00:23:04.584975 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:04.584060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.129:36863)
0811 00:23:04.584599 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:04.584776 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":257,"threads_started":1}
I20260811 00:23:05.077075 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:05.076182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.130:47837)
0811 00:23:05.076753 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:05.076873 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":264,"threads_started":1}
I20260811 00:23:05.528555 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 11196
I20260811 00:23:05.550608 12102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:05.530114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.16.71.129:45615)
0811 00:23:05.530832 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:05.530855 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:05.530909 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:23:05.533094 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:05.533108 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:05.533599 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:05.533924 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:05.535776 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:05.537402 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:05.538913 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:05.539746 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:05.539948 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:05.543118 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:05.543129 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:05.543150 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:05.543210 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:05.546558 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:05.547509 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:05.547525 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:05.547535 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:05.547747 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:05.548186 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:05.548190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:05.548192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:05.548975 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:05.549317 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:05.549842 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 00:23:05.550308 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":327,"threads_started":1}
I20260811 00:23:05.552085 11509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:05.559180 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:05.558307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:36785 (local address 127.16.71.131:54399)
0811 00:23:05.558841 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:05.558944 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":268,"threads_started":1}
I20260811 00:23:05.638767 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 11196
I20260811 00:23:05.667520 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 11521
I20260811 00:23:05.687530 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:05.668702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.16.71.130:38641)
0811 00:23:05.669326 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:05.669370 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:05.669411 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:23:05.671102 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:05.671115 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:05.671614 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:05.671959 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:05.673633 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:05.674987 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:05.676514 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:05.677356 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:05.677539 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:05.680511 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:05.680521 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:05.680540 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:05.680587 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:05.683746 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:05.684599 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:05.684607 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:05.684645 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:05.684912 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:05.685357 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:05.685361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:05.685363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:05.685999 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:05.686244 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:05.686711 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 00:23:05.687220 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":324,"threads_started":1}
I20260811 00:23:05.689150 11759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:05.774914 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 11521
I20260811 00:23:05.802006 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 11770
I20260811 00:23:05.821874 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:05.803108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.16.71.131:45973)
0811 00:23:05.803751 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:05.803765 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:05.803809 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:23:05.805628 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:05.805641 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:05.806122 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:05.806486 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:05.808017 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:05.809434 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:05.810851 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:05.811598 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:05.811759 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:05.814905 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:05.814914 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:05.814934 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:05.814993 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:05.818202 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:05.819183 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:05.819191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:05.819198 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:05.819404 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:05.819767 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:05.819771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:05.819773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:05.820379 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:05.820663 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:05.821104 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 00:23:05.821557 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":303,"threads_started":1}
I20260811 00:23:05.823457 12005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:05.909440 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 11770
2026-08-11T00:23:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25799 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:23:06 dist-test-slave-f7th krb5kdc[12118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:23:06 dist-test-slave-f7th krb5kdc[12118](info): set up 2 sockets
Aug 11 00:23:06 dist-test-slave-f7th krb5kdc[12118](info): commencing operation
krb5kdc: starting...
W20260811 00:23:07.220023 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:23:07 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:23:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:23:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:23:07.548981 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45917
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:23:07.918918 12138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:07.919294 12138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:07.919426 12138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:07.927215 12138 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:23:07.927353 12138 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:23:07.927417 12138 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:23:07.927491 12138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:23:07.927569 12138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:07.927666 12138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:07.927729 12138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:23:07.927809 12138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:23:07.938493 12138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:45917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:07.940042 12138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:07.941567 12138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:07.956110 12150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:07.956157 12153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:07.956177 12149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:07.956789 12138 server_base.cc:1034] running on GCE node
I20260811 00:23:07.957746 12138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:07.959476 12138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:07.960774 12138 hybrid_clock.cc:648] HybridClock initialized: now 1786407787960670 us; error 73 us; skew 500 ppm
I20260811 00:23:07.961165 12138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:07 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407787, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:07.965137 12138 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:23:07.967051 12138 webserver.cc:460] Webserver started at http://127.16.71.190:34673/ using document root <none> and password file <none>
I20260811 00:23:07.967875 12138 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:07.967984 12138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:07.968307 12138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:23:07.971863 12138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "ab6383571e9743eaa6f2a038cb9649a4"
format_stamp: "Formatted at 2026-08-11 00:23:07 on dist-test-slave-f7th"
server_key: "f3c516028175c33b8250de72b436b8e7"
server_key_iv: "0001a39e9d58e10a3a73e12c674da83c"
server_key_version: "encryptionkey@0"
I20260811 00:23:07.972806 12138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "ab6383571e9743eaa6f2a038cb9649a4"
format_stamp: "Formatted at 2026-08-11 00:23:07 on dist-test-slave-f7th"
server_key: "f3c516028175c33b8250de72b436b8e7"
server_key_iv: "0001a39e9d58e10a3a73e12c674da83c"
server_key_version: "encryptionkey@0"
I20260811 00:23:07.979454 12138 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:23:07.984011 12163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:07.986565 12138 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 00:23:07.987067 12138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "ab6383571e9743eaa6f2a038cb9649a4"
format_stamp: "Formatted at 2026-08-11 00:23:07 on dist-test-slave-f7th"
server_key: "f3c516028175c33b8250de72b436b8e7"
server_key_iv: "0001a39e9d58e10a3a73e12c674da83c"
server_key_version: "encryptionkey@0"
I20260811 00:23:07.987375 12138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:08.005334 12138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:08.009979 12138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:08.010248 12138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:08.029479 12138 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:45917
I20260811 00:23:08.029531 12259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:45917 every 8 connection(s)
I20260811 00:23:08.031016 12138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:23:08.032354 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 12138
I20260811 00:23:08.032601 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:23:08.033449 16670 external_mini_cluster.cc:1428] Setting key d9ef3c28ab5fe911a87af4589e1c92cd
I20260811 00:23:08.035715 12260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:23:08.050101 12260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Bootstrap starting.
I20260811 00:23:08.053511 12260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Neither blocks nor log segments found. Creating new log.
I20260811 00:23:08.054703 12260 log.cc:824] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:23:08 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:23:08.058530 12260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: No bootstrap required, opened a new log
I20260811 00:23:08.069751 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:08.035671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.16.71.190:45917)
0811 00:23:08.036583 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:08.036599 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:08.036686 (+ 87us) server_negotiation.cc:407] Connection header received
0811 00:23:08.038933 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:08.038969 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:08.039573 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:08.040101 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:08.041800 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:08.043894 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:08.045543 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:08.046659 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:08.046930 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:08.059181 (+ 12251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:08.059195 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:08.059223 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:08.059285 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:08.063911 (+ 4626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:08.064994 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:08.065007 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:08.065021 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:08.065292 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:08.065792 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:08.065798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:08.065802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:08.066669 (+ 867us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:08.067076 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:08.067563 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 00:23:08.068155 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":378,"threads_started":1}
I20260811 00:23:08.069730 12260 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:08.070277 12260 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:23:08.070384 12260 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6383571e9743eaa6f2a038cb9649a4, State: Initialized, Role: FOLLOWER
I20260811 00:23:08.071000 12260 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:08.071242 12260 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:23:08.071434 12260 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:23:08.071619 12260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:23:08.074174 12260 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:08.074716 12260 leader_election.cc:302] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab6383571e9743eaa6f2a038cb9649a4; no voters:
I20260811 00:23:08.075347 12260 leader_election.cc:288] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:23:08.075532 12268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:23:08.076665 12268 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 LEADER]: Becoming Leader. State: Replica: ab6383571e9743eaa6f2a038cb9649a4, State: Running, Role: LEADER
I20260811 00:23:08.077337 12268 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:08.077718 12260 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:23:08.080847 12271 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab6383571e9743eaa6f2a038cb9649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } } }
I20260811 00:23:08.081003 12272 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab6383571e9743eaa6f2a038cb9649a4. Latest consensus state: current_term: 1 leader_uuid: "ab6383571e9743eaa6f2a038cb9649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } } }
I20260811 00:23:08.082387 12271 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:23:08.082587 12272 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:23:08.083493 12282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:23:08.094823 12282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:23:08.105197 12282 catalog_manager.cc:1347] Generated new cluster ID: 9b0fca54f3b649fba3b16596d02845cb
I20260811 00:23:08.105340 12282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:23:08.122244 12282 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:23:08.124408 12282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:23:08.135820 12282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Generated new TSK 0
I20260811 00:23:08.136895 12282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:23:08.216749 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45917
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:23:08.583557 12307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:08.583904 12307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:08.584048 12307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:08.591872 12307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:08.592036 12307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:08.592195 12307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:23:08.602747 12307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:08.604192 12307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:08.605630 12307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:08.621073 12313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:08.621115 12323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:08.621146 12316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:08.622108 12307 server_base.cc:1034] running on GCE node
I20260811 00:23:08.622833 12307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:08.623989 12307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:08.625247 12307 hybrid_clock.cc:648] HybridClock initialized: now 1786407788625159 us; error 83 us; skew 500 ppm
I20260811 00:23:08.625643 12307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:08 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407788, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:08.629340 12307 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:23:08.631031 12307 webserver.cc:460] Webserver started at http://127.16.71.129:39217/ using document root <none> and password file <none>
I20260811 00:23:08.631800 12307 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:08.631937 12307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:08.632252 12307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:23:08.635449 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9"
format_stamp: "Formatted at 2026-08-11 00:23:08 on dist-test-slave-f7th"
server_key: "8195b21e634b2e4ea61f3cc7aafef82a"
server_key_iv: "5928e605c368ca860dbed1c3df74df63"
server_key_version: "encryptionkey@0"
I20260811 00:23:08.636292 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9"
format_stamp: "Formatted at 2026-08-11 00:23:08 on dist-test-slave-f7th"
server_key: "8195b21e634b2e4ea61f3cc7aafef82a"
server_key_iv: "5928e605c368ca860dbed1c3df74df63"
server_key_version: "encryptionkey@0"
I20260811 00:23:08.641914 12307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:23:08.645766 12333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:08.647274 12307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:23:08.647468 12307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9"
format_stamp: "Formatted at 2026-08-11 00:23:08 on dist-test-slave-f7th"
server_key: "8195b21e634b2e4ea61f3cc7aafef82a"
server_key_iv: "5928e605c368ca860dbed1c3df74df63"
server_key_version: "encryptionkey@0"
I20260811 00:23:08.647680 12307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:08.679903 12307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:08.684720 12307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:08.684993 12307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:08.686023 12307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:08.687481 12307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:08.687572 12307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:08.687672 12307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:08.687716 12307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:08.719970 12307 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:46443
I20260811 00:23:08.720024 12536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:46443 every 8 connection(s)
I20260811 00:23:08.721413 12307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:23:08.731401 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 12307
I20260811 00:23:08.731599 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:23:08.732311 16670 external_mini_cluster.cc:1428] Setting key abbf9834496104648c3516ed80d4d200
Aug 11 00:23:08 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407788, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:23:08.742152 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:08.724146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:34213 (local address 127.16.71.190:45917)
0811 00:23:08.724607 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:08.724643 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:08.725519 (+ 876us) server_negotiation.cc:407] Connection header received
0811 00:23:08.727456 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:08.727462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:08.727554 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:08.727664 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:08.729394 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:08.730127 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:08.731828 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:08.732252 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:08.732375 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:08.736052 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:08.736058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:08.736061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:08.736102 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:08.738730 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:08.739579 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:08.739584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:08.739586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:08.739718 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:08.740183 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:08.740187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:08.740189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:08.740487 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:08.740652 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:08.741493 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 00:23:08.741761 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":262,"threads_started":1}
I20260811 00:23:08.743432 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:08.724572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:34213)
0811 00:23:08.725346 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:08.725386 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:08.727148 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:08.727946 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:08.727956 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:08.728506 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:23:08.729203 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:08.729222 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:08.730310 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:08.730315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:08.731673 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:08.731691 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:08.732515 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:08.732522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:08.732792 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:08.733468 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:23:08.733492 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:23:08.735788 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:23:08.738913 (+ 3125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:08.738925 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:08.738943 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:08.739436 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:08.739832 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:08.739838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:08.739842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:08.740059 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:08.740649 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:23:08.740657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:23:08.741055 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 00:23:08.741341 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 00:23:08.742019 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":291,"threads_started":1}
I20260811 00:23:08.745683 12538 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45917
I20260811 00:23:08.746122 12538 heartbeater.cc:464] Registering TS with master...
I20260811 00:23:08.747152 12538 heartbeater.cc:511] Master 127.16.71.190:45917 requested a full tablet report, sending...
I20260811 00:23:08.749763 12199 ts_manager.cc:194] Registered new tserver with Master: 9ae8a8cc6e8343e699a46c8c0daa41e9 (127.16.71.129:46443)
I20260811 00:23:08.751358 12199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:34213
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:23:08.802140 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45917
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:23:09.167222 12550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:09.167552 12550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:09.167675 12550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:09.175812 12550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:09.175958 12550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:09.176110 12550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:23:09.186806 12550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:09.188262 12550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:09.189709 12550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:09.203446 12563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:09.203575 12569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:09.203636 12566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:09.204316 12550 server_base.cc:1034] running on GCE node
I20260811 00:23:09.205019 12550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:09.206040 12550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:09.207374 12550 hybrid_clock.cc:648] HybridClock initialized: now 1786407789207297 us; error 53 us; skew 500 ppm
I20260811 00:23:09.207764 12550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:09 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407789, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:09.211308 12550 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:23:09.213120 12550 webserver.cc:460] Webserver started at http://127.16.71.130:37315/ using document root <none> and password file <none>
I20260811 00:23:09.213876 12550 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:09.214025 12550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:09.214411 12550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:23:09.217753 12550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "88e2276d75594bad8fbe5602365c55f6"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "6c1cbe4ecc78f1e7ac4b93d4899ea886"
server_key_iv: "f38b2495a9b8d6f3be46185c127ad318"
server_key_version: "encryptionkey@0"
I20260811 00:23:09.218616 12550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "88e2276d75594bad8fbe5602365c55f6"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "6c1cbe4ecc78f1e7ac4b93d4899ea886"
server_key_iv: "f38b2495a9b8d6f3be46185c127ad318"
server_key_version: "encryptionkey@0"
I20260811 00:23:09.224749 12550 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 00:23:09.228948 12581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:09.230726 12550 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:23:09.230924 12550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "88e2276d75594bad8fbe5602365c55f6"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "6c1cbe4ecc78f1e7ac4b93d4899ea886"
server_key_iv: "f38b2495a9b8d6f3be46185c127ad318"
server_key_version: "encryptionkey@0"
I20260811 00:23:09.231163 12550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:09.248848 12550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:09.255147 12550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:09.255484 12550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:09.256736 12550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:09.258219 12550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:09.258306 12550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:09.258430 12550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:09.258486 12550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:09.291990 12550 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:37675
I20260811 00:23:09.292009 12767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:37675 every 8 connection(s)
I20260811 00:23:09.293895 12550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:23:09.294384 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 12550
I20260811 00:23:09.294600 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:23:09.295257 16670 external_mini_cluster.cc:1428] Setting key 46369464e652dbcd8661b9fea3b482ac
Aug 11 00:23:09 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407789, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:23:09.315992 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.296850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:38277 (local address 127.16.71.190:45917)
0811 00:23:09.297283 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:09.297287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:09.298573 (+ 1286us) server_negotiation.cc:407] Connection header received
0811 00:23:09.300499 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:09.300504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:09.300596 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:09.300748 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:09.302593 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.303343 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.305119 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.305555 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.305676 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.309603 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:09.309609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:09.309612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:09.309649 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:09.312355 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:09.313372 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:09.313376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:09.313379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:09.313524 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:09.314032 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:09.314035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:09.314037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:09.314393 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:09.314514 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:09.315512 (+ 998us) server_negotiation.cc:299] Negotiation successful
0811 00:23:09.315706 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":234,"threads_started":1}
I20260811 00:23:09.317680 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.297573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:38277)
0811 00:23:09.298409 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:09.298457 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:09.300189 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:09.301008 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:09.301018 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:09.301737 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:23:09.302391 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.302411 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.303543 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.303547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.304956 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.304971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.305801 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.305808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.306051 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.306752 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:23:09.306776 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:23:09.309319 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:23:09.312535 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:09.312545 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:09.312564 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:09.313202 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:09.313657 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:09.313661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:09.313664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:09.313935 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:09.314524 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:23:09.314533 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:23:09.315012 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 00:23:09.315384 (+ 372us) client_negotiation.cc:240] Negotiation successful
0811 00:23:09.315986 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":350,"threads_started":1}
I20260811 00:23:09.319515 12769 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45917
I20260811 00:23:09.319852 12769 heartbeater.cc:464] Registering TS with master...
I20260811 00:23:09.320736 12769 heartbeater.cc:511] Master 127.16.71.190:45917 requested a full tablet report, sending...
I20260811 00:23:09.322433 12199 ts_manager.cc:194] Registered new tserver with Master: 88e2276d75594bad8fbe5602365c55f6 (127.16.71.130:37675)
I20260811 00:23:09.323330 12199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:38277
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:23:09.364563 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45917
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:23:09.728048 12780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:09.728432 12780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:09.728663 12780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:09.736704 12780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:09.736869 12780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:09.737051 12780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:23:09.747514 12780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:09.749058 12780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:09.750519 12780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 00:23:09.755633 12538 heartbeater.cc:503] Master 127.16.71.190:45917 was elected leader, sending a full tablet report...
W20260811 00:23:09.765306 12793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:09.765334 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:09.765376 12791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:09.766309 12780 server_base.cc:1034] running on GCE node
I20260811 00:23:09.767262 12780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:09.768666 12780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:09.769960 12780 hybrid_clock.cc:648] HybridClock initialized: now 1786407789769900 us; error 78 us; skew 500 ppm
I20260811 00:23:09.770357 12780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:09 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407789, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:09.774047 12780 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:23:09.775938 12780 webserver.cc:460] Webserver started at http://127.16.71.131:44971/ using document root <none> and password file <none>
I20260811 00:23:09.776767 12780 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:09.776906 12780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:09.777208 12780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:23:09.780807 12780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "d88c03b839b34bafb52dae8016f07be2"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "018023eef7b31ac8e6a331239e71b0bc"
server_key_iv: "98eaf1cc7ec5ccdcea75bbd33d5e2fe2"
server_key_version: "encryptionkey@0"
I20260811 00:23:09.781750 12780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "d88c03b839b34bafb52dae8016f07be2"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "018023eef7b31ac8e6a331239e71b0bc"
server_key_iv: "98eaf1cc7ec5ccdcea75bbd33d5e2fe2"
server_key_version: "encryptionkey@0"
I20260811 00:23:09.788599 12780 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:23:09.793582 12807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:09.795392 12780 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:23:09.795635 12780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "d88c03b839b34bafb52dae8016f07be2"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "018023eef7b31ac8e6a331239e71b0bc"
server_key_iv: "98eaf1cc7ec5ccdcea75bbd33d5e2fe2"
server_key_version: "encryptionkey@0"
I20260811 00:23:09.795943 12780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:09.812357 12780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:09.818086 12780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:09.818423 12780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:09.819536 12780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:09.821142 12780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:09.821228 12780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:09.821350 12780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:09.821409 12780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:09.855804 12780 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:36913
I20260811 00:23:09.855829 13006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:36913 every 8 connection(s)
I20260811 00:23:09.857496 12780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:23:09.867681 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 12780
I20260811 00:23:09.867908 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:23:09.868360 16670 external_mini_cluster.cc:1428] Setting key 2baa09c4dd9930e2cc891b09b45b9a96
Aug 11 00:23:09 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407789, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:23:09.881803 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.860480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:36091 (local address 127.16.71.190:45917)
0811 00:23:09.861027 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:09.861032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:09.862049 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 00:23:09.864118 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:09.864124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:09.864235 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:09.864341 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:09.866272 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.867143 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.869545 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.870213 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.870336 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.874810 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:09.874816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:09.874820 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:09.874859 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:09.877605 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:09.878522 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:09.878526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:09.878529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:09.878663 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:09.879298 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:09.879306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:09.879308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:09.880047 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:09.880222 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:09.881249 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0811 00:23:09.881474 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":300,"threads_started":1}
I20260811 00:23:09.882929 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.860963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:36091)
0811 00:23:09.861813 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:09.861859 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:09.863742 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:09.864701 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:09.864713 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:09.865337 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:23:09.866029 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.866051 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.867356 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.867360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.869334 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.869357 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.870560 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.870569 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.870916 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.871780 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:23:09.871819 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:23:09.874516 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:23:09.877810 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:09.877820 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:09.877837 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:09.878385 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:09.878769 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:09.878773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:09.878775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:09.879002 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:09.880240 (+ 1238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:23:09.880248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:23:09.880772 (+ 524us) client_negotiation.cc:769] Sending connection context
0811 00:23:09.881061 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 00:23:09.881544 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":330,"threads_started":1}
I20260811 00:23:09.884725 13008 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45917
I20260811 00:23:09.885149 13008 heartbeater.cc:464] Registering TS with master...
I20260811 00:23:09.885970 13008 heartbeater.cc:511] Master 127.16.71.190:45917 requested a full tablet report, sending...
I20260811 00:23:09.887727 12199 ts_manager.cc:194] Registered new tserver with Master: d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913)
I20260811 00:23:09.888566 12199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:36091
I20260811 00:23:09.897802 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:23:09.916882 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.902730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.16.71.190:45917)
0811 00:23:09.903023 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:09.903031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:09.903454 (+ 423us) server_negotiation.cc:407] Connection header received
0811 00:23:09.903643 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:09.903647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:09.903732 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:09.903824 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:09.905536 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.906318 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.907745 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.908156 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.908278 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.910983 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:09.910989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:09.910992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:09.911031 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:09.913924 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:09.914707 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:09.914711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:09.914714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:09.914858 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:09.915250 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:09.915255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:09.915258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:09.915679 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:09.915933 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:09.916289 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 00:23:09.916533 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:23:09.921383 12199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:23:09.928759 12199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:23:09.966264 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.942599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36913 (local address 127.0.0.1:37658)
0811 00:23:09.954697 (+ 12098us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:09.954744 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:09.954956 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:09.956711 (+ 1755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:09.956719 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:09.956759 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:09.957334 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.957351 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.960605 (+ 3254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.960610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.962866 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.962882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.965019 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.965031 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.965265 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.965330 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 00:23:09.965483 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 00:23:09.965732 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11762,"mutex_wait_us":66,"thread_start_us":14032,"threads_started":1}
I20260811 00:23:09.967059 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.942766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.16.71.131:36913)
0811 00:23:09.955875 (+ 13109us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:09.955886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:09.955925 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:23:09.956054 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:09.956063 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:09.956369 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:09.957371 (+ 1002us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:09.957690 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.960425 (+ 2735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.963069 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.964403 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.964550 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.964877 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:09.966365 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0811 00:23:09.966676 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12892,"thread_start_us":12704,"threads_started":1}
I20260811 00:23:09.967077 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.943492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.16.71.129:46443)
0811 00:23:09.953863 (+ 10371us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:09.953873 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:09.957490 (+ 3617us) server_negotiation.cc:407] Connection header received
0811 00:23:09.957750 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:09.957757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:09.957945 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:09.958112 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:09.958866 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.960933 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.963749 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.965010 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.965172 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.965405 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:09.965993 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 00:23:09.966197 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10179,"thread_start_us":10391,"threads_started":1}
I20260811 00:23:09.967166 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.942757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:37675 (local address 127.0.0.1:43858)
0811 00:23:09.957042 (+ 14285us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:09.957065 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:09.957346 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:09.958326 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:09.958335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:09.958361 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:09.958817 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.958830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.960830 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.960835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.963498 (+ 2663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.963516 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.965627 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.965633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.965816 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.965847 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:23:09.966620 (+ 773us) client_negotiation.cc:240] Negotiation successful
0811 00:23:09.966767 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14095,"mutex_wait_us":110,"thread_start_us":14284,"threads_started":1}
I20260811 00:23:09.967262 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.943326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.16.71.130:37675)
0811 00:23:09.953322 (+ 9996us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:09.953333 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:09.957209 (+ 3876us) server_negotiation.cc:407] Connection header received
0811 00:23:09.957573 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:09.957582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:09.957832 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:09.958034 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:09.959016 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.960592 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.963702 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.965203 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.965344 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.966055 (+ 711us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:09.966716 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 00:23:09.966927 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9774,"thread_start_us":9623,"threads_started":1}
I20260811 00:23:09.967939 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:09.943080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46443 (local address 127.0.0.1:55796)
0811 00:23:09.957354 (+ 14274us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:09.957372 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:09.957581 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:09.958128 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:09.958133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:09.958159 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:09.958597 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.958616 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.961107 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.961110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.963502 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:09.963516 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:09.965572 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:09.965580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:09.965755 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:09.965786 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:23:09.967412 (+ 1626us) client_negotiation.cc:240] Negotiation successful
0811 00:23:09.967571 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14133,"mutex_wait_us":33,"thread_start_us":11315,"threads_started":1}
I20260811 00:23:09.971930 12430 tablet_service.cc:1467] Processing CreateTablet for tablet 91773e2ad5654489a1b13ebddc734414 (DEFAULT_TABLE table=test-table [id=93363a09e94e48e7bbb130deee43230c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:23:09.971916 12894 tablet_service.cc:1467] Processing CreateTablet for tablet 91773e2ad5654489a1b13ebddc734414 (DEFAULT_TABLE table=test-table [id=93363a09e94e48e7bbb130deee43230c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:23:09.971935 12656 tablet_service.cc:1467] Processing CreateTablet for tablet 91773e2ad5654489a1b13ebddc734414 (DEFAULT_TABLE table=test-table [id=93363a09e94e48e7bbb130deee43230c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:23:09.973580 12894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91773e2ad5654489a1b13ebddc734414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:23:09.973615 12430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91773e2ad5654489a1b13ebddc734414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:23:09.973615 12656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91773e2ad5654489a1b13ebddc734414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:23:09.986163 13038 tablet_bootstrap.cc:492] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Bootstrap starting.
I20260811 00:23:09.990427 13038 tablet_bootstrap.cc:654] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Neither blocks nor log segments found. Creating new log.
I20260811 00:23:09.991673 13038 log.cc:824] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:09.993238 13039 tablet_bootstrap.cc:492] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Bootstrap starting.
I20260811 00:23:09.994740 13040 tablet_bootstrap.cc:492] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Bootstrap starting.
I20260811 00:23:09.995635 13038 tablet_bootstrap.cc:492] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: No bootstrap required, opened a new log
I20260811 00:23:09.995924 13038 ts_tablet_manager.cc:1397] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 00:23:09.998919 13040 tablet_bootstrap.cc:654] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Neither blocks nor log segments found. Creating new log.
I20260811 00:23:09.998909 13039 tablet_bootstrap.cc:654] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Neither blocks nor log segments found. Creating new log.
I20260811 00:23:10.000977 13039 log.cc:824] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:10.000976 13040 log.cc:824] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:10.005228 13039 tablet_bootstrap.cc:492] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: No bootstrap required, opened a new log
I20260811 00:23:10.005584 13039 ts_tablet_manager.cc:1397] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260811 00:23:10.005607 13040 tablet_bootstrap.cc:492] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: No bootstrap required, opened a new log
I20260811 00:23:10.005934 13040 ts_tablet_manager.cc:1397] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260811 00:23:10.006332 13038 raft_consensus.cc:348] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.006923 13038 raft_consensus.cc:374] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:23:10.007089 13038 raft_consensus.cc:729] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d88c03b839b34bafb52dae8016f07be2, State: Initialized, Role: FOLLOWER
I20260811 00:23:10.007844 13038 consensus_queue.cc:260] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.009733 13008 heartbeater.cc:503] Master 127.16.71.190:45917 was elected leader, sending a full tablet report...
I20260811 00:23:10.009944 13038 ts_tablet_manager.cc:1428] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 00:23:10.015483 13039 raft_consensus.cc:348] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.015612 13040 raft_consensus.cc:348] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.016077 13039 raft_consensus.cc:374] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:23:10.016250 13039 raft_consensus.cc:729] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae8a8cc6e8343e699a46c8c0daa41e9, State: Initialized, Role: FOLLOWER
I20260811 00:23:10.016465 13040 raft_consensus.cc:374] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:23:10.016681 13040 raft_consensus.cc:729] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e2276d75594bad8fbe5602365c55f6, State: Initialized, Role: FOLLOWER
I20260811 00:23:10.017128 13039 consensus_queue.cc:260] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.017385 13040 consensus_queue.cc:260] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.018657 12769 heartbeater.cc:503] Master 127.16.71.190:45917 was elected leader, sending a full tablet report...
I20260811 00:23:10.019085 13040 ts_tablet_manager.cc:1428] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Time spent starting tablet: real 0.013s user 0.001s sys 0.012s
I20260811 00:23:10.021064 13039 ts_tablet_manager.cc:1428] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260811 00:23:10.042104 13056 raft_consensus.cc:482] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:23:10.042467 13056 raft_consensus.cc:504] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.044219 13056 leader_election.cc:288] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913), 9ae8a8cc6e8343e699a46c8c0daa41e9 (127.16.71.129:46443)
W20260811 00:23:10.046365 12772 tablet.cc:2369] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:23:10.052966 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.044873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:54439 (local address 127.16.71.131:36913)
0811 00:23:10.045120 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:10.045126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:10.045926 (+ 800us) server_negotiation.cc:407] Connection header received
0811 00:23:10.046120 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:10.046125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:10.046194 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:10.046277 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:10.047262 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.048079 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.050681 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.051899 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.052086 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.052166 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:10.052493 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 00:23:10.052701 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:23:10.053064 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.044705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:38933 (local address 127.16.71.129:46443)
0811 00:23:10.044933 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:10.044939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:10.045873 (+ 934us) server_negotiation.cc:407] Connection header received
0811 00:23:10.046083 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:10.046089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:10.046170 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:10.046277 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:10.047154 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.047937 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.050673 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.051947 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.052103 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.052166 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:10.052668 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 00:23:10.052818 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:23:10.053224 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.044657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36913 (local address 127.16.71.130:54439)
0811 00:23:10.045721 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:10.045761 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:10.045975 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:10.046463 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:10.046471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:10.046522 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:10.047088 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:10.047104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.048227 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.048236 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:10.050487 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:10.050506 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.052136 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.052141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:10.052288 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.052348 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 00:23:10.052432 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 00:23:10.052570 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":75,"spinlock_wait_cycles":44928,"thread_start_us":523,"threads_started":1}
I20260811 00:23:10.053268 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.044680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46443 (local address 127.16.71.130:38933)
0811 00:23:10.045749 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:10.045763 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:10.045931 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:10.046326 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:10.046331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:10.046364 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:10.046937 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:10.046956 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.048073 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.048078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:10.050487 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:10.050506 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.052336 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.052340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:10.052483 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.052516 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:23:10.052607 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:23:10.052770 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":118,"thread_start_us":547,"threads_started":1}
I20260811 00:23:10.054352 12936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91773e2ad5654489a1b13ebddc734414" candidate_uuid: "88e2276d75594bad8fbe5602365c55f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d88c03b839b34bafb52dae8016f07be2" is_pre_election: true
I20260811 00:23:10.054350 12471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91773e2ad5654489a1b13ebddc734414" candidate_uuid: "88e2276d75594bad8fbe5602365c55f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" is_pre_election: true
I20260811 00:23:10.054874 12936 raft_consensus.cc:2454] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88e2276d75594bad8fbe5602365c55f6 in term 0.
I20260811 00:23:10.054873 12471 raft_consensus.cc:2454] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88e2276d75594bad8fbe5602365c55f6 in term 0.
I20260811 00:23:10.055521 12585 leader_election.cc:302] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e2276d75594bad8fbe5602365c55f6, d88c03b839b34bafb52dae8016f07be2; no voters:
I20260811 00:23:10.055928 13056 raft_consensus.cc:2781] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:23:10.056093 13056 raft_consensus.cc:482] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:23:10.056195 13056 raft_consensus.cc:3037] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:23:10.058435 13056 raft_consensus.cc:504] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.059100 13056 leader_election.cc:288] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [CANDIDATE]: Term 1 election: Requested vote from peers d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913), 9ae8a8cc6e8343e699a46c8c0daa41e9 (127.16.71.129:46443)
I20260811 00:23:10.059624 12936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91773e2ad5654489a1b13ebddc734414" candidate_uuid: "88e2276d75594bad8fbe5602365c55f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d88c03b839b34bafb52dae8016f07be2"
I20260811 00:23:10.059691 12471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91773e2ad5654489a1b13ebddc734414" candidate_uuid: "88e2276d75594bad8fbe5602365c55f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9"
I20260811 00:23:10.059900 12471 raft_consensus.cc:3037] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:23:10.059901 12936 raft_consensus.cc:3037] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:23:10.063256 12936 raft_consensus.cc:2454] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88e2276d75594bad8fbe5602365c55f6 in term 1.
I20260811 00:23:10.063256 12471 raft_consensus.cc:2454] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88e2276d75594bad8fbe5602365c55f6 in term 1.
I20260811 00:23:10.063858 12583 leader_election.cc:302] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 88e2276d75594bad8fbe5602365c55f6, 9ae8a8cc6e8343e699a46c8c0daa41e9, d88c03b839b34bafb52dae8016f07be2; no voters:
I20260811 00:23:10.064234 13056 raft_consensus.cc:2781] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:23:10.064750 13056 raft_consensus.cc:686] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 1 LEADER]: Becoming Leader. State: Replica: 88e2276d75594bad8fbe5602365c55f6, State: Running, Role: LEADER
I20260811 00:23:10.065297 13056 consensus_queue.cc:237] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:10.070070 12194 catalog_manager.cc:5495] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88e2276d75594bad8fbe5602365c55f6 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "88e2276d75594bad8fbe5602365c55f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } health_report { overall_health: UNKNOWN } } }
I20260811 00:23:10.087282 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.081521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.16.71.130:37675)
0811 00:23:10.081752 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:10.081758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:10.081778 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:23:10.081992 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:10.081996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:10.082102 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:10.082251 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:23:10.082678 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.083431 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.085042 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.085449 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.085574 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.086070 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:23:10.086606 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:23:10.086738 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:10.086863 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 00:23:10.086992 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:23:10.096696 12471 raft_consensus.cc:1261] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 FOLLOWER]: Refusing update from remote peer 88e2276d75594bad8fbe5602365c55f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:23:10.096696 12936 raft_consensus.cc:1261] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Refusing update from remote peer 88e2276d75594bad8fbe5602365c55f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:23:10.098085 13056 consensus_queue.cc:1035] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:23:10.098445 13060 consensus_queue.cc:1035] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:23:10.109407 13010 tablet.cc:2369] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:23:10.114530 13071 mvcc.cc:205] Tried to move back new op lower bound from 7317126308223131648 to 7317126308111204352. Current Snapshot: MvccSnapshot[applied={T|T < 7317126308223131648}]
I20260811 00:23:10.116701 13068 mvcc.cc:205] Tried to move back new op lower bound from 7317126308223131648 to 7317126308111204352. Current Snapshot: MvccSnapshot[applied={T|T < 7317126308223131648}]
I20260811 00:23:10.128959 12194 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:23:10.129297 12194 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:23:10.133680 12194 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Sending DeleteTablet for 3 replicas of tablet 91773e2ad5654489a1b13ebddc734414
I20260811 00:23:10.135306 12430 tablet_service.cc:1514] Processing DeleteTablet for tablet 91773e2ad5654489a1b13ebddc734414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:23:10 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:55796
I20260811 00:23:10.135542 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 12307
I20260811 00:23:10.136687 12656 tablet_service.cc:1514] Processing DeleteTablet for tablet 91773e2ad5654489a1b13ebddc734414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:23:10 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:43858
I20260811 00:23:10.137893 12894 tablet_service.cc:1514] Processing DeleteTablet for tablet 91773e2ad5654489a1b13ebddc734414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:23:10 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:37658
I20260811 00:23:10.146688 13085 tablet_replica.cc:330] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: stopping tablet replica
I20260811 00:23:10.147351 13085 raft_consensus.cc:2229] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:23:10.148028 13085 raft_consensus.cc:2258] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:23:10.150750 13085 ts_tablet_manager.cc:1905] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:23:10 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:23:10.158690 13085 ts_tablet_manager.cc:1918] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:23:10.158933 13085 log.cc:1196] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/91773e2ad5654489a1b13ebddc734414
I20260811 00:23:10.160006 13085 ts_tablet_manager.cc:1939] T 91773e2ad5654489a1b13ebddc734414 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Deleting consensus metadata
I20260811 00:23:10.161525 13090 tablet_replica.cc:330] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: stopping tablet replica
I20260811 00:23:10.162621 13088 tablet_replica.cc:330] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: stopping tablet replica
I20260811 00:23:10.162884 12167 catalog_manager.cc:4841] TS 9ae8a8cc6e8343e699a46c8c0daa41e9 (127.16.71.129:46443): tablet 91773e2ad5654489a1b13ebddc734414 (table test-table [id=93363a09e94e48e7bbb130deee43230c]) successfully deleted
I20260811 00:23:10.163609 13090 raft_consensus.cc:2229] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:23:10.163697 13088 raft_consensus.cc:2229] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:23:10.164273 13090 raft_consensus.cc:2258] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:23:10.164505 13088 raft_consensus.cc:2258] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:23:10.167148 13090 ts_tablet_manager.cc:1905] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:23:10.167160 13088 ts_tablet_manager.cc:1905] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:23:10.170233 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.137494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.16.71.129:46443)
0811 00:23:10.137696 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:10.137703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:10.146365 (+ 8662us) server_negotiation.cc:407] Connection header received
0811 00:23:10.146839 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:10.146846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:10.146942 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:10.147027 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:10.149403 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.150502 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.152771 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.153445 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.153551 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.160858 (+ 7307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:10.160873 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:10.160893 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:10.160942 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:10.165795 (+ 4853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:10.167055 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:10.167067 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:10.167071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:10.167274 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:10.167849 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:10.167854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:10.167857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:10.168398 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:10.168765 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:10.169514 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 00:23:10.169819 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:23:10.172374 12531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:10.175609 13090 ts_tablet_manager.cc:1918] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:23:10.175958 13090 log.cc:1196] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/91773e2ad5654489a1b13ebddc734414
I20260811 00:23:10.175951 13088 ts_tablet_manager.cc:1918] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:23:10.176250 13088 log.cc:1196] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/91773e2ad5654489a1b13ebddc734414
I20260811 00:23:10.176785 13090 ts_tablet_manager.cc:1939] T 91773e2ad5654489a1b13ebddc734414 P d88c03b839b34bafb52dae8016f07be2: Deleting consensus metadata
I20260811 00:23:10.177055 13088 ts_tablet_manager.cc:1939] T 91773e2ad5654489a1b13ebddc734414 P 88e2276d75594bad8fbe5602365c55f6: Deleting consensus metadata
I20260811 00:23:10.179457 12169 catalog_manager.cc:4841] TS d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913): tablet 91773e2ad5654489a1b13ebddc734414 (table test-table [id=93363a09e94e48e7bbb130deee43230c]) successfully deleted
I20260811 00:23:10.179872 12171 catalog_manager.cc:4841] TS 88e2276d75594bad8fbe5602365c55f6 (127.16.71.130:37675): tablet 91773e2ad5654489a1b13ebddc734414 (table test-table [id=93363a09e94e48e7bbb130deee43230c]) successfully deleted
I20260811 00:23:10.333118 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 12307
I20260811 00:23:10.362793 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 12550
Aug 11 00:23:10 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:23:10.380270 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.363722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.16.71.130:37675)
0811 00:23:10.363937 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:10.363943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:10.363963 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:23:10.364021 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:10.364025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:10.364108 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:10.364196 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:10.365648 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.366595 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.367986 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.368440 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.368563 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.373264 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:10.373280 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:10.373301 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:10.373352 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:10.376863 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:10.377885 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:10.377894 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:10.377897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:10.378081 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:10.378515 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:10.378519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:10.378521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:10.378974 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:10.379320 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:10.379648 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 00:23:10.379894 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:23:10.381165 12763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:10.498531 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 12550
I20260811 00:23:10.521610 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 12780
Aug 11 00:23:10 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:23:10.538245 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:10.522370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37668 (local address 127.16.71.131:36913)
0811 00:23:10.522559 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:10.522566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:10.522623 (+ 57us) server_negotiation.cc:407] Connection header received
0811 00:23:10.522815 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:10.522819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:10.522902 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:10.523002 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:10.524488 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.525309 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.526686 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:10.527114 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:10.527233 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:10.532075 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:10.532094 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:10.532112 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:10.532154 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:10.535055 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:10.535900 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:10.535907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:10.535910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:10.536098 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:10.536576 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:10.536580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:10.536582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:10.537108 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:10.537365 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:10.537760 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 00:23:10.537974 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:23:10.539081 12997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:10.629113 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 12780
I20260811 00:23:10.656526 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 12138
I20260811 00:23:10.657850 12250 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:10.770879 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 12138
I20260811 00:23:30.809484 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45917
--webserver_interface=127.16.71.190
--webserver_port=34673
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:23:31.182010 13098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:31.182332 13098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:31.182461 13098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:31.190037 13098 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:23:31.190151 13098 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:23:31.190191 13098 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:23:31.190258 13098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:23:31.190341 13098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:31.190426 13098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:31.190481 13098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:23:31.190536 13098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:23:31.201611 13098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:45917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=34673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:31.203105 13098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:31.204564 13098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:31.231513 13107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:31.231429 13108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:31.231480 13110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:31.234565 13098 server_base.cc:1034] running on GCE node
I20260811 00:23:31.235658 13098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:31.237540 13098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:31.238857 13098 hybrid_clock.cc:648] HybridClock initialized: now 1786407811238777 us; error 55 us; skew 500 ppm
I20260811 00:23:31.239324 13098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:31 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:31.248569 13098 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:23:31.251924 13098 webserver.cc:460] Webserver started at http://127.16.71.190:34673/ using document root <none> and password file <none>
I20260811 00:23:31.253227 13098 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:31.253410 13098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:31.261421 13098 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:23:31.265166 13124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:31.267092 13098 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:23:31.267349 13098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "ab6383571e9743eaa6f2a038cb9649a4"
format_stamp: "Formatted at 2026-08-11 00:23:07 on dist-test-slave-f7th"
server_key: "f3c516028175c33b8250de72b436b8e7"
server_key_iv: "0001a39e9d58e10a3a73e12c674da83c"
server_key_version: "encryptionkey@0"
I20260811 00:23:31.268580 13098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:31.290124 13098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:31.295547 13098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:31.295889 13098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:31.316300 13098 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:45917
I20260811 00:23:31.316323 13207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:45917 every 8 connection(s)
I20260811 00:23:31.318030 13098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:23:31.323311 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 13098
I20260811 00:23:31.324353 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:46443
--local_ip_for_outbound_sockets=127.16.71.129
--tserver_master_addrs=127.16.71.190:45917
--webserver_port=39217
--webserver_interface=127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:23:31.326880 13208 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:23:31.331982 13208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Bootstrap starting.
I20260811 00:23:31.354939 13208 log.cc:824] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:31.370795 13208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:23:31.371404 13208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Bootstrap complete.
I20260811 00:23:31.382272 13208 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:31.383294 13208 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6383571e9743eaa6f2a038cb9649a4, State: Initialized, Role: FOLLOWER
I20260811 00:23:31.384024 13208 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:31.384270 13208 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:23:31.384402 13208 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:23:31.384577 13208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:23:31.389093 13208 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:31.389767 13208 leader_election.cc:302] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab6383571e9743eaa6f2a038cb9649a4; no voters:
I20260811 00:23:31.390676 13208 leader_election.cc:288] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:23:31.390879 13215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:23:31.392089 13215 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 LEADER]: Becoming Leader. State: Replica: ab6383571e9743eaa6f2a038cb9649a4, State: Running, Role: LEADER
I20260811 00:23:31.392802 13215 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:31.393481 13208 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:23:31.396148 13222 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab6383571e9743eaa6f2a038cb9649a4. Latest consensus state: current_term: 2 leader_uuid: "ab6383571e9743eaa6f2a038cb9649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } } }
I20260811 00:23:31.396641 13222 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:23:31.397222 13218 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ab6383571e9743eaa6f2a038cb9649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } } }
I20260811 00:23:31.397619 13218 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:23:31.410612 13227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:23:31.419747 13227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:23:31.421478 13227 catalog_manager.cc:1259] Loaded cluster ID: 9b0fca54f3b649fba3b16596d02845cb
I20260811 00:23:31.421622 13227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:23:31.425994 13227 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:23:31.428275 13227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Loaded TSK: 0
I20260811 00:23:31.462889 13227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Generated new TSK 1
I20260811 00:23:31.463406 13227 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:23:31.738642 13212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:31.738981 13212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:31.739137 13212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:31.746939 13212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:31.747102 13212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:31.747267 13212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:23:31.758332 13212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:46443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=39217
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:31.759820 13212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:31.761274 13212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:31.783025 13265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:31.782995 13268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:31.782949 13272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:31.785374 13212 server_base.cc:1034] running on GCE node
I20260811 00:23:31.786145 13212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:31.787380 13212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:31.788762 13212 hybrid_clock.cc:648] HybridClock initialized: now 1786407811788683 us; error 67 us; skew 500 ppm
I20260811 00:23:31.789252 13212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:31 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:31.793232 13212 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:23:31.795159 13212 webserver.cc:460] Webserver started at http://127.16.71.129:39217/ using document root <none> and password file <none>
I20260811 00:23:31.795980 13212 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:31.796140 13212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:31.802968 13212 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:23:31.806456 13284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:31.808557 13212 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:23:31.808804 13212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9"
format_stamp: "Formatted at 2026-08-11 00:23:08 on dist-test-slave-f7th"
server_key: "8195b21e634b2e4ea61f3cc7aafef82a"
server_key_iv: "5928e605c368ca860dbed1c3df74df63"
server_key_version: "encryptionkey@0"
I20260811 00:23:31.809995 13212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:31.835280 13212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:31.840269 13212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:31.840579 13212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:31.841749 13212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:31.843291 13212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:31.843412 13212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:31.843530 13212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:31.843601 13212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:31.878563 13212 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:46443
I20260811 00:23:31.878621 13475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:46443 every 8 connection(s)
I20260811 00:23:31.880115 13212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:23:31.881603 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 13212
I20260811 00:23:31.882396 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:37675
--local_ip_for_outbound_sockets=127.16.71.130
--tserver_master_addrs=127.16.71.190:45917
--webserver_port=37315
--webserver_interface=127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:23:31 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:23:31.906531 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:31.883642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:35845 (local address 127.16.71.190:45917)
0811 00:23:31.884330 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:31.884345 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:31.885084 (+ 739us) server_negotiation.cc:407] Connection header received
0811 00:23:31.887346 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:31.887359 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:31.887731 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:31.888113 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:31.889867 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:31.891166 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:31.892958 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:31.893656 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:31.893853 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:31.897855 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:31.897865 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:31.897883 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:31.897938 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:31.901389 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:31.902332 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:31.902340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:31.902346 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:31.902555 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:31.903053 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:31.903057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:31.903059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:31.903610 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:31.903815 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:31.904715 (+ 900us) server_negotiation.cc:299] Negotiation successful
0811 00:23:31.905140 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":331,"threads_started":1}
I20260811 00:23:31.906533 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:31.884005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:35845)
0811 00:23:31.884890 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:31.884932 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:31.886728 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:31.888302 (+ 1574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:31.888312 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:31.889026 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:23:31.889608 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:31.889628 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:31.891375 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:31.891379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:31.892771 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:31.892789 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:31.893934 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:31.893943 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:31.894237 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:31.894943 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:23:31.894976 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:23:31.897477 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:23:31.901599 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:31.901608 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:31.901621 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:31.902131 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:31.902695 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:31.902699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:31.902701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:31.902915 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:31.903776 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:23:31.903784 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:23:31.904140 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 00:23:31.904407 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 00:23:31.905130 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":326,"threads_started":1}
I20260811 00:23:31.910104 13476 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45917
I20260811 00:23:31.910440 13476 heartbeater.cc:464] Registering TS with master...
I20260811 00:23:31.911288 13476 heartbeater.cc:511] Master 127.16.71.190:45917 requested a full tablet report, sending...
I20260811 00:23:31.914095 13158 ts_manager.cc:194] Registered new tserver with Master: 9ae8a8cc6e8343e699a46c8c0daa41e9 (127.16.71.129:46443)
I20260811 00:23:31.916430 13158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:35845
W20260811 00:23:32.251117 13486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:32.251456 13486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:32.251554 13486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:32.259163 13486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:32.259303 13486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:32.259420 13486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:23:32.269317 13486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:37675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=37315
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:32.270622 13486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:32.271936 13486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:32.285750 13496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:32.285748 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:32.285701 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:32.286705 13486 server_base.cc:1034] running on GCE node
I20260811 00:23:32.287441 13486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:32.288646 13486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:32.289953 13486 hybrid_clock.cc:648] HybridClock initialized: now 1786407812289898 us; error 45 us; skew 500 ppm
I20260811 00:23:32.290337 13486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:32 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:32.294008 13486 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:23:32.295737 13486 webserver.cc:460] Webserver started at http://127.16.71.130:37315/ using document root <none> and password file <none>
I20260811 00:23:32.296536 13486 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:32.296715 13486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:32.302717 13486 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:23:32.306098 13515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:32.308156 13486 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:23:32.308385 13486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "88e2276d75594bad8fbe5602365c55f6"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "6c1cbe4ecc78f1e7ac4b93d4899ea886"
server_key_iv: "f38b2495a9b8d6f3be46185c127ad318"
server_key_version: "encryptionkey@0"
I20260811 00:23:32.309609 13486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:32.322901 13486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:32.327594 13486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:32.327898 13486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:32.329030 13486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:32.330550 13486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:32.330657 13486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:32.330775 13486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:32.330854 13486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:32.364145 13486 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:37675
I20260811 00:23:32.364204 13722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:37675 every 8 connection(s)
I20260811 00:23:32.365790 13486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:23:32.376042 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 13486
I20260811 00:23:32.376776 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:36913
--local_ip_for_outbound_sockets=127.16.71.131
--tserver_master_addrs=127.16.71.190:45917
--webserver_port=44971
--webserver_interface=127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:23:32 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:23:32.390422 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.368694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:55203 (local address 127.16.71.190:45917)
0811 00:23:32.368921 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.368926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.370135 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 00:23:32.372449 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.372456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.372609 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.372745 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:32.375018 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.375907 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.378311 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.378932 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.379082 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.382990 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:32.383000 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:32.383005 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:32.383053 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:32.386272 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.387314 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.387321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.387324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.387519 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.388056 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.388060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.388063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.388458 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:32.388657 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:32.389767 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0811 00:23:32.390073 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:23:32.391324 13727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.369101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:55203)
0811 00:23:32.369916 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:32.369968 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:32.372042 (+ 2074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:32.373036 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:32.373051 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:32.373913 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:23:32.374800 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.374826 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.376174 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.376181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.378097 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.378116 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.379215 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.379222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.379457 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.380211 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:23:32.380237 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:23:32.382661 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:23:32.386491 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:32.386500 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:32.386518 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:32.387111 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:32.387665 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:32.387669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:32.387671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:32.387889 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:32.388703 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:23:32.388712 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:23:32.389234 (+ 522us) client_negotiation.cc:769] Sending connection context
0811 00:23:32.389594 (+ 360us) client_negotiation.cc:240] Negotiation successful
0811 00:23:32.390090 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":327,"threads_started":1}
I20260811 00:23:32.393008 13725 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45917
I20260811 00:23:32.393404 13725 heartbeater.cc:464] Registering TS with master...
I20260811 00:23:32.394328 13725 heartbeater.cc:511] Master 127.16.71.190:45917 requested a full tablet report, sending...
I20260811 00:23:32.396314 13158 ts_manager.cc:194] Registered new tserver with Master: 88e2276d75594bad8fbe5602365c55f6 (127.16.71.130:37675)
I20260811 00:23:32.397462 13158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:55203
W20260811 00:23:32.749131 13729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:32.749454 13729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:32.749553 13729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:32.757220 13729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:32.757360 13729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:32.757476 13729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:23:32.767989 13729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:36913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=44971
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:32.769497 13729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:32.770982 13729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:32.786732 13738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:32.786700 13741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:32.786659 13739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:32.787621 13729 server_base.cc:1034] running on GCE node
I20260811 00:23:32.788326 13729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:32.789484 13729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:32.790753 13729 hybrid_clock.cc:648] HybridClock initialized: now 1786407812790682 us; error 59 us; skew 500 ppm
I20260811 00:23:32.791172 13729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:32 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:32.795111 13729 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:23:32.796973 13729 webserver.cc:460] Webserver started at http://127.16.71.131:44971/ using document root <none> and password file <none>
I20260811 00:23:32.797870 13729 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:32.798038 13729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:32.804550 13729 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 00:23:32.807816 13756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:32.809540 13729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:23:32.809751 13729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "d88c03b839b34bafb52dae8016f07be2"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "018023eef7b31ac8e6a331239e71b0bc"
server_key_iv: "98eaf1cc7ec5ccdcea75bbd33d5e2fe2"
server_key_version: "encryptionkey@0"
I20260811 00:23:32.810932 13729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:32.830476 13729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:32.835819 13729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:32.836320 13729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:32.837635 13729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:32.839357 13729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:32.839479 13729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:32.839617 13729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:32.839699 13729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:32.874596 13729 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:36913
I20260811 00:23:32.874697 13959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:36913 every 8 connection(s)
I20260811 00:23:32.876139 13729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:23:32.885190 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 13729
Aug 11 00:23:32 dist-test-slave-f7th krb5kdc[12118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:23:32.900166 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.879421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:53941 (local address 127.16.71.190:45917)
0811 00:23:32.879611 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.879616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.881236 (+ 1620us) server_negotiation.cc:407] Connection header received
0811 00:23:32.883293 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.883301 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.883395 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.883489 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:32.885471 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.886362 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.888288 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.888838 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.888968 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.893078 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:32.893089 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:32.893093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:32.893128 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:32.896094 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.897113 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.897118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.897121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.897283 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.897865 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.897870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.897872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.898217 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:32.898367 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:32.899441 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0811 00:23:32.899764 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:23:32.901443 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.879974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:53941)
0811 00:23:32.881023 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:32.881092 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:32.882964 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:32.883765 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:32.883774 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:32.884340 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:23:32.885204 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.885236 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.886645 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.886652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.888105 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.888120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.889105 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.889115 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.889422 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.890179 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:23:32.890205 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:23:32.892836 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:23:32.896298 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:32.896307 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:32.896325 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:32.896957 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:32.897422 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:23:32.897427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:23:32.897431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:23:32.897725 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:23:32.898367 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:23:32.898377 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:23:32.898794 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 00:23:32.899253 (+ 459us) client_negotiation.cc:240] Negotiation successful
0811 00:23:32.899981 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":377,"threads_started":1}
I20260811 00:23:32.901918 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.886214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47358 (local address 127.16.71.190:45917)
0811 00:23:32.887043 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.887048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.887065 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:23:32.887109 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.887112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.887168 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.887256 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:32.888913 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.889729 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.891206 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.891853 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.891997 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.895762 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:32.895768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:32.895771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:32.895822 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:32.898716 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.899515 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.899519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.899522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.899653 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.900189 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.900193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.900196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.900592 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:32.900911 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:32.901296 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 00:23:32.901556 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":543,"threads_started":1}
I20260811 00:23:32.903254 13961 heartbeater.cc:347] Connected to a master server at 127.16.71.190:45917
I20260811 00:23:32.903645 13961 heartbeater.cc:464] Registering TS with master...
I20260811 00:23:32.904505 13961 heartbeater.cc:511] Master 127.16.71.190:45917 requested a full tablet report, sending...
I20260811 00:23:32.906551 13158 ts_manager.cc:194] Registered new tserver with Master: d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913)
I20260811 00:23:32.907527 13158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:53941
I20260811 00:23:32.918699 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:23:32.920432 13476 heartbeater.cc:503] Master 127.16.71.190:45917 was elected leader, sending a full tablet report...
I20260811 00:23:32.926738 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.919903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.16.71.190:45917)
0811 00:23:32.920159 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.920167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.920732 (+ 565us) server_negotiation.cc:407] Connection header received
0811 00:23:32.921067 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.921074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.921251 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.921436 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:23:32.922038 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.922851 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.924592 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.925123 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.925232 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.925944 (+ 712us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:23:32.926188 (+ 244us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 00:23:32.926523 (+ 335us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47362: token expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 00:23:32.926904 13967 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47362: token expired
W20260811 00:23:32.927567 13013 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.16.71.190:45917)
I20260811 00:23:32.942543 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.928133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47378 (local address 127.16.71.190:45917)
0811 00:23:32.928332 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.928337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.928357 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:23:32.928469 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.928475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.928538 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.928665 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:32.930230 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.930998 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.932650 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.933147 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.933278 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.936265 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:32.936272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:32.936275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:32.936318 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:32.939307 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.940243 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.940248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.940250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.940418 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:32.940900 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:32.940904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:32.940913 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:32.941303 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:32.941560 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:32.941959 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 00:23:32.942207 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:23:32.947247 13158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:23:32.956286 13158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:23:32.999336 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.972597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.16.71.131:36913)
0811 00:23:32.989820 (+ 17223us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.989829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.989867 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:23:32.989986 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.989995 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.990306 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.990566 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:32.991438 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.992553 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.996343 (+ 3790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.997992 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.998142 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.998433 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:32.998769 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 00:23:32.999010 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17041,"thread_start_us":16853,"threads_started":1}
I20260811 00:23:32.999353 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.971833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36913 (local address 127.0.0.1:60780)
0811 00:23:32.987493 (+ 15660us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:32.987520 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:32.987700 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:32.990612 (+ 2912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:32.990620 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:32.990652 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:32.991194 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.991208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.992765 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.992771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.995857 (+ 3086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.995875 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.998248 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.998256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.998485 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.998551 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 00:23:32.998676 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 00:23:32.998911 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15479,"mutex_wait_us":1,"thread_start_us":9840,"threads_started":1}
I20260811 00:23:33.000497 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.971445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:37675 (local address 127.0.0.1:42226)
0811 00:23:32.982304 (+ 10859us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:32.982366 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:32.982602 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:32.990834 (+ 8232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:32.990838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:32.990882 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:32.992914 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.992936 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.994781 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.994785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.997378 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.997396 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.999516 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.999524 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.999698 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:32.999736 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:23:32.999812 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 00:23:32.999975 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10433,"mutex_wait_us":76,"thread_start_us":15662,"threads_started":1}
I20260811 00:23:33.001199 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.971665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.16.71.130:37675)
0811 00:23:32.990250 (+ 18585us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.990256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.990283 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:23:32.990366 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.990372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.990574 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.991480 (+ 906us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:32.993309 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.994630 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.997615 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.999217 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.000325 (+ 1108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.000540 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.000685 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.000873 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18426,"thread_start_us":18319,"threads_started":1}
I20260811 00:23:33.001861 13998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.972872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.16.71.129:46443)
0811 00:23:32.989363 (+ 16491us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:32.989374 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:32.990167 (+ 793us) server_negotiation.cc:407] Connection header received
0811 00:23:32.990272 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:32.990279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:32.990475 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:32.990644 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:32.993681 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.995473 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.998290 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.999806 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.999947 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.000202 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.001334 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.001549 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16295,"thread_start_us":16144,"threads_started":1}
I20260811 00:23:33.002317 13994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:32.972007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46443 (local address 127.0.0.1:52020)
0811 00:23:32.989867 (+ 17860us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:32.989888 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:32.990034 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:32.990924 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:32.990933 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:32.990962 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:32.993077 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.993092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:32.995647 (+ 2555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:32.995653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:32.998041 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:32.998058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.000088 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.000096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:33.000279 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.000313 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:23:33.000405 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 00:23:33.000564 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17717,"mutex_wait_us":1,"thread_start_us":17550,"threads_started":1}
I20260811 00:23:33.004739 13839 tablet_service.cc:1467] Processing CreateTablet for tablet 4cf228f508b544bcb55d6e45f10e00b1 (DEFAULT_TABLE table=test-table [id=0e475edb82904107a9fb2e4abb701c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:23:33.005648 13366 tablet_service.cc:1467] Processing CreateTablet for tablet 4cf228f508b544bcb55d6e45f10e00b1 (DEFAULT_TABLE table=test-table [id=0e475edb82904107a9fb2e4abb701c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:23:33.005671 13615 tablet_service.cc:1467] Processing CreateTablet for tablet 4cf228f508b544bcb55d6e45f10e00b1 (DEFAULT_TABLE table=test-table [id=0e475edb82904107a9fb2e4abb701c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:23:33.006531 13839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cf228f508b544bcb55d6e45f10e00b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:23:33.007055 13615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cf228f508b544bcb55d6e45f10e00b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:23:33.007514 13366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cf228f508b544bcb55d6e45f10e00b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:23:33.022847 13999 tablet_bootstrap.cc:492] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Bootstrap starting.
I20260811 00:23:33.026387 13999 tablet_bootstrap.cc:654] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Neither blocks nor log segments found. Creating new log.
I20260811 00:23:33.027424 14000 tablet_bootstrap.cc:492] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Bootstrap starting.
I20260811 00:23:33.027932 13999 log.cc:824] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:33.028966 14001 tablet_bootstrap.cc:492] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Bootstrap starting.
I20260811 00:23:33.031701 13999 tablet_bootstrap.cc:492] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: No bootstrap required, opened a new log
I20260811 00:23:33.032030 13999 ts_tablet_manager.cc:1397] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 00:23:33.032898 14001 tablet_bootstrap.cc:654] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Neither blocks nor log segments found. Creating new log.
I20260811 00:23:33.032893 14000 tablet_bootstrap.cc:654] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Neither blocks nor log segments found. Creating new log.
I20260811 00:23:33.034339 14000 log.cc:824] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:33.034406 14001 log.cc:824] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:33.038301 14001 tablet_bootstrap.cc:492] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: No bootstrap required, opened a new log
I20260811 00:23:33.038301 14000 tablet_bootstrap.cc:492] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: No bootstrap required, opened a new log
I20260811 00:23:33.038667 14001 ts_tablet_manager.cc:1397] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260811 00:23:33.038672 14000 ts_tablet_manager.cc:1397] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260811 00:23:33.040776 13999 raft_consensus.cc:348] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.041391 13999 raft_consensus.cc:374] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:23:33.041545 13999 raft_consensus.cc:729] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae8a8cc6e8343e699a46c8c0daa41e9, State: Initialized, Role: FOLLOWER
I20260811 00:23:33.042332 13999 consensus_queue.cc:260] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.044437 13999 ts_tablet_manager.cc:1428] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Time spent starting tablet: real 0.012s user 0.012s sys 0.002s
I20260811 00:23:33.050344 14001 raft_consensus.cc:348] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.050344 14000 raft_consensus.cc:348] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.050913 14000 raft_consensus.cc:374] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:23:33.050915 14001 raft_consensus.cc:374] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:23:33.051043 14001 raft_consensus.cc:729] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d88c03b839b34bafb52dae8016f07be2, State: Initialized, Role: FOLLOWER
I20260811 00:23:33.051036 14000 raft_consensus.cc:729] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e2276d75594bad8fbe5602365c55f6, State: Initialized, Role: FOLLOWER
I20260811 00:23:33.051780 14001 consensus_queue.cc:260] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.051779 14000 consensus_queue.cc:260] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.053109 13961 heartbeater.cc:503] Master 127.16.71.190:45917 was elected leader, sending a full tablet report...
I20260811 00:23:33.053637 13725 heartbeater.cc:503] Master 127.16.71.190:45917 was elected leader, sending a full tablet report...
I20260811 00:23:33.054020 14001 ts_tablet_manager.cc:1428] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260811 00:23:33.055625 14000 ts_tablet_manager.cc:1428] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Time spent starting tablet: real 0.017s user 0.006s sys 0.011s
W20260811 00:23:33.117893 13726 tablet.cc:2369] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:23:33.128250 13964 tablet.cc:2369] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:23:33.132952 13479 tablet.cc:2369] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:23:33.142091 14007 raft_consensus.cc:482] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:23:33.142459 14007 raft_consensus.cc:504] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.144318 14007 leader_election.cc:288] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88e2276d75594bad8fbe5602365c55f6 (127.16.71.130:37675), d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913)
I20260811 00:23:33.152000 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.144906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:56825 (local address 127.16.71.130:37675)
0811 00:23:33.145138 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:33.145145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:33.145905 (+ 760us) server_negotiation.cc:407] Connection header received
0811 00:23:33.146124 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:33.146131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:33.146197 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:33.146275 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:33.147077 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.147969 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.150213 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.151041 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.151177 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.151236 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.151603 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.151743 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:23:33.152104 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.144923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:37675 (local address 127.16.71.129:56825)
0811 00:23:33.145733 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:33.145764 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:33.145936 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:33.146317 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:33.146322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:33.146347 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:33.146877 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:33.146896 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.148151 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.148157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:33.150013 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:33.150029 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.151264 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.151269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:33.151406 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.151436 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:23:33.151535 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 00:23:33.151695 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":103,"thread_start_us":407,"threads_started":1}
I20260811 00:23:33.153224 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.145127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36913 (local address 127.16.71.129:60607)
0811 00:23:33.145832 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:33.145849 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:23:33.145995 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:23:33.146468 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:23:33.146472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:23:33.146512 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:23:33.146947 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:33.146964 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.148190 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.148196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:33.150003 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:23:33.150023 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.152458 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.152465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:23:33.152680 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.152717 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:23:33.152819 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 00:23:33.152940 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":36,"thread_start_us":426,"threads_started":1}
I20260811 00:23:33.153338 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.144912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:60607 (local address 127.16.71.131:36913)
0811 00:23:33.145143 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:33.145150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:33.145972 (+ 822us) server_negotiation.cc:407] Connection header received
0811 00:23:33.146147 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:33.146152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:33.146227 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:33.146315 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:23:33.147152 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.148046 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.151327 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.152159 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.152331 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.152415 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.152877 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.153080 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:23:33.153265 13646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cf228f508b544bcb55d6e45f10e00b1" candidate_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e2276d75594bad8fbe5602365c55f6" is_pre_election: true
I20260811 00:23:33.153862 13646 raft_consensus.cc:2454] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae8a8cc6e8343e699a46c8c0daa41e9 in term 0.
I20260811 00:23:33.154467 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cf228f508b544bcb55d6e45f10e00b1" candidate_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d88c03b839b34bafb52dae8016f07be2" is_pre_election: true
I20260811 00:23:33.154829 13291 leader_election.cc:302] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e2276d75594bad8fbe5602365c55f6, 9ae8a8cc6e8343e699a46c8c0daa41e9; no voters:
I20260811 00:23:33.155027 13872 raft_consensus.cc:2454] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae8a8cc6e8343e699a46c8c0daa41e9 in term 0.
I20260811 00:23:33.155287 14007 raft_consensus.cc:2781] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:23:33.155443 14007 raft_consensus.cc:482] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:23:33.155570 14007 raft_consensus.cc:3037] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:23:33.157928 14007 raft_consensus.cc:504] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.158703 14007 leader_election.cc:288] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [CANDIDATE]: Term 1 election: Requested vote from peers 88e2276d75594bad8fbe5602365c55f6 (127.16.71.130:37675), d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913)
I20260811 00:23:33.159279 13646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cf228f508b544bcb55d6e45f10e00b1" candidate_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e2276d75594bad8fbe5602365c55f6"
I20260811 00:23:33.159421 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cf228f508b544bcb55d6e45f10e00b1" candidate_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d88c03b839b34bafb52dae8016f07be2"
I20260811 00:23:33.159561 13646 raft_consensus.cc:3037] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:23:33.159646 13872 raft_consensus.cc:3037] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:23:33.162997 13872 raft_consensus.cc:2454] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae8a8cc6e8343e699a46c8c0daa41e9 in term 1.
I20260811 00:23:33.162997 13646 raft_consensus.cc:2454] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae8a8cc6e8343e699a46c8c0daa41e9 in term 1.
I20260811 00:23:33.163548 13291 leader_election.cc:302] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e2276d75594bad8fbe5602365c55f6, 9ae8a8cc6e8343e699a46c8c0daa41e9; no voters:
I20260811 00:23:33.163858 14007 raft_consensus.cc:2781] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:23:33.164353 14007 raft_consensus.cc:686] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae8a8cc6e8343e699a46c8c0daa41e9, State: Running, Role: LEADER
I20260811 00:23:33.164913 14007 consensus_queue.cc:237] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } }
I20260811 00:23:33.170003 13158 catalog_manager.cc:5495] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ae8a8cc6e8343e699a46c8c0daa41e9 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46443 } health_report { overall_health: HEALTHY } } }
I20260811 00:23:33.198742 13998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.192578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.16.71.129:46443)
0811 00:23:33.192832 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:33.192839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:33.192964 (+ 125us) server_negotiation.cc:407] Connection header received
0811 00:23:33.193232 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:33.193236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:33.193330 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:33.193540 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:23:33.193968 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.194745 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.196203 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.196660 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.196788 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.197387 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:23:33.197974 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:23:33.198121 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.198317 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.198452 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:23:33.209486 13872 raft_consensus.cc:1261] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Refusing update from remote peer 9ae8a8cc6e8343e699a46c8c0daa41e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:23:33.209477 13646 raft_consensus.cc:1261] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 1 FOLLOWER]: Refusing update from remote peer 9ae8a8cc6e8343e699a46c8c0daa41e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:23:33.210839 14014 consensus_queue.cc:1035] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d88c03b839b34bafb52dae8016f07be2" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:23:33.211324 14007 consensus_queue.cc:1035] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88e2276d75594bad8fbe5602365c55f6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:23:33.227694 14018 mvcc.cc:205] Tried to move back new op lower bound from 7317126402892652544 to 7317126402728050688. Current Snapshot: MvccSnapshot[applied={T|T < 7317126402892652544}]
I20260811 00:23:33.249431 13158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:23:33.249840 13158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:23:33.254962 13158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Sending DeleteTablet for 3 replicas of tablet 4cf228f508b544bcb55d6e45f10e00b1
I20260811 00:23:33.257611 13366 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cf228f508b544bcb55d6e45f10e00b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:23:33 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:52020
I20260811 00:23:33.257936 13839 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cf228f508b544bcb55d6e45f10e00b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:23:33 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:60780
I20260811 00:23:33.258723 14033 tablet_replica.cc:330] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: stopping tablet replica
I20260811 00:23:33.259238 13615 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cf228f508b544bcb55d6e45f10e00b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:23:33 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:42226
I20260811 00:23:33.259435 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 13212
I20260811 00:23:33.260326 14041 tablet_replica.cc:330] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: stopping tablet replica
I20260811 00:23:33.260921 14041 raft_consensus.cc:2229] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:23:33.261660 14041 raft_consensus.cc:2258] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:23:33.264312 14041 ts_tablet_manager.cc:1905] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:23:33.269658 14037 tablet_replica.cc:330] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: stopping tablet replica
I20260811 00:23:33.270202 14037 raft_consensus.cc:2229] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:23:33.270802 14037 raft_consensus.cc:2258] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:23:33.272295 14041 ts_tablet_manager.cc:1918] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:23:33.272576 14041 log.cc:1196] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/4cf228f508b544bcb55d6e45f10e00b1
I20260811 00:23:33.273315 14041 ts_tablet_manager.cc:1939] T 4cf228f508b544bcb55d6e45f10e00b1 P 88e2276d75594bad8fbe5602365c55f6: Deleting consensus metadata
I20260811 00:23:33.275619 13131 catalog_manager.cc:4841] TS 88e2276d75594bad8fbe5602365c55f6 (127.16.71.130:37675): tablet 4cf228f508b544bcb55d6e45f10e00b1 (table test-table [id=0e475edb82904107a9fb2e4abb701c5f]) successfully deleted
I20260811 00:23:33.277161 14033 raft_consensus.cc:2229] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:23:33.278712 14033 raft_consensus.cc:2258] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:23:33.282739 14033 ts_tablet_manager.cc:1905] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:23:33.285759 14037 ts_tablet_manager.cc:1905] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:23:33.290828 14033 ts_tablet_manager.cc:1918] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:23:33.291177 14033 log.cc:1196] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/4cf228f508b544bcb55d6e45f10e00b1
I20260811 00:23:33.291839 14033 ts_tablet_manager.cc:1939] T 4cf228f508b544bcb55d6e45f10e00b1 P 9ae8a8cc6e8343e699a46c8c0daa41e9: Deleting consensus metadata
I20260811 00:23:33.294265 13998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.261094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52040 (local address 127.16.71.129:46443)
0811 00:23:33.261288 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:33.261294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:33.261318 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:23:33.261384 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:33.261389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:33.261510 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:33.261618 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:33.263912 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.265131 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.267202 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.267942 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.268077 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.272377 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:33.272397 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:33.272418 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:33.272472 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:33.277127 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:33.278092 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:33.278107 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:33.278111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:33.278345 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:33.279932 (+ 1587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:33.279938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:33.279942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:33.280444 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:33.280811 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.293536 (+ 12725us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.293870 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:23:33.295476 13470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:33.295921 14037 ts_tablet_manager.cc:1918] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:23:33.296128 14037 log.cc:1196] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/4cf228f508b544bcb55d6e45f10e00b1
I20260811 00:23:33.296876 14037 ts_tablet_manager.cc:1939] T 4cf228f508b544bcb55d6e45f10e00b1 P d88c03b839b34bafb52dae8016f07be2: Deleting consensus metadata
I20260811 00:23:33.303404 13130 catalog_manager.cc:4841] TS d88c03b839b34bafb52dae8016f07be2 (127.16.71.131:36913): tablet 4cf228f508b544bcb55d6e45f10e00b1 (table test-table [id=0e475edb82904107a9fb2e4abb701c5f]) successfully deleted
I20260811 00:23:33.405279 13129 catalog_manager.cc:4841] TS 9ae8a8cc6e8343e699a46c8c0daa41e9 (127.16.71.129:46443): tablet 4cf228f508b544bcb55d6e45f10e00b1 (table test-table [id=0e475edb82904107a9fb2e4abb701c5f]) successfully deleted
I20260811 00:23:33.405795 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 13212
I20260811 00:23:33.434159 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 13486
I20260811 00:23:33.450706 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.435100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42230 (local address 127.16.71.130:37675)
0811 00:23:33.435318 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:33.435324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:33.435348 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:23:33.435411 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:33.435415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:33.435481 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:33.435574 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:33.437122 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.437956 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.439622 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.440040 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.440161 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.443235 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:33.443255 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:33.443274 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:33.443323 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:33.447048 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:33.448138 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:33.448149 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:33.448151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:33.448347 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:33.448991 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:33.448996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:33.448998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:33.449491 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:33.449765 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.450134 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.450351 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:23:33.451460 13716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:33.544301 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 13486
I20260811 00:23:33.574435 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 13729
I20260811 00:23:33.589614 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:33.575199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.16.71.131:36913)
0811 00:23:33.575361 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 00:23:33.575365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:23:33.575385 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:23:33.575577 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:23:33.575581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:23:33.575654 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:23:33.575769 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:23:33.577561 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.578329 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.579888 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:23:33.580431 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:23:33.580555 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:23:33.583564 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:23:33.583581 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:23:33.583600 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:23:33.583641 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:23:33.586604 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:33.587506 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:33.587517 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:33.587519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:33.587675 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:23:33.588118 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:23:33.588121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:23:33.588123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:23:33.588478 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:23:33.588757 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 00:23:33.589158 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 00:23:33.589338 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 00:23:33.590262 13952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:33.681089 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 13729
I20260811 00:23:33.708603 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 13098
I20260811 00:23:33.709939 13203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:23:33.826128 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 13098
I20260811 00:23:53.867337 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45917
--webserver_interface=127.16.71.190
--webserver_port=34673
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:23:54.245626 14049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:54.246058 14049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:54.246182 14049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:54.253849 14049 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:23:54.253979 14049 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:23:54.254024 14049 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:23:54.254079 14049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:23:54.254134 14049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:54.254204 14049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:54.254251 14049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:23:54.254305 14049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:23:54.264940 14049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:45917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=34673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:54.266314 14049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:54.267692 14049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:54.280948 14060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:54.280849 14056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:54.281169 14049 server_base.cc:1034] running on GCE node
W20260811 00:23:54.280900 14058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:54.282354 14049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:54.284212 14049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:54.285478 14049 hybrid_clock.cc:648] HybridClock initialized: now 1786407834285401 us; error 51 us; skew 500 ppm
I20260811 00:23:54.285871 14049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:54 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407834, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:54.289376 14049 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:23:54.291085 14049 webserver.cc:460] Webserver started at http://127.16.71.190:34673/ using document root <none> and password file <none>
I20260811 00:23:54.291873 14049 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:54.292006 14049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:54.298444 14049 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 00:23:54.302134 14074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:54.303891 14049 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:23:54.304124 14049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "ab6383571e9743eaa6f2a038cb9649a4"
format_stamp: "Formatted at 2026-08-11 00:23:07 on dist-test-slave-f7th"
server_key: "f3c516028175c33b8250de72b436b8e7"
server_key_iv: "0001a39e9d58e10a3a73e12c674da83c"
server_key_version: "encryptionkey@0"
I20260811 00:23:54.305338 14049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:54.324236 14049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:54.329160 14049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:54.329476 14049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:54.349519 14049 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:45917
I20260811 00:23:54.349568 14169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:45917 every 8 connection(s)
I20260811 00:23:54.351225 14049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:23:54.358919 14170 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:23:54.361955 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 14049
I20260811 00:23:54.363114 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:46443
--local_ip_for_outbound_sockets=127.16.71.129
--tserver_master_addrs=127.16.71.190:45917
--webserver_port=39217
--webserver_interface=127.16.71.129
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:23:54.363219 14170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Bootstrap starting.
I20260811 00:23:54.391613 14170 log.cc:824] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:23:54.411949 14170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:23:54.412521 14170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: Bootstrap complete.
I20260811 00:23:54.422582 14170 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:54.423604 14170 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6383571e9743eaa6f2a038cb9649a4, State: Initialized, Role: FOLLOWER
I20260811 00:23:54.424333 14170 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:54.424603 14170 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:23:54.424767 14170 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:23:54.424957 14170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:23:54.429294 14170 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:54.429862 14170 leader_election.cc:302] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab6383571e9743eaa6f2a038cb9649a4; no voters:
I20260811 00:23:54.430505 14170 leader_election.cc:288] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 00:23:54.430738 14177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:23:54.431789 14177 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [term 3 LEADER]: Becoming Leader. State: Replica: ab6383571e9743eaa6f2a038cb9649a4, State: Running, Role: LEADER
I20260811 00:23:54.432608 14170 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:23:54.432561 14177 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } }
I20260811 00:23:54.439517 14181 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab6383571e9743eaa6f2a038cb9649a4. Latest consensus state: current_term: 3 leader_uuid: "ab6383571e9743eaa6f2a038cb9649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } } }
I20260811 00:23:54.439939 14181 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:23:54.439007 14178 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ab6383571e9743eaa6f2a038cb9649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6383571e9743eaa6f2a038cb9649a4" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45917 } } }
I20260811 00:23:54.440567 14178 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:23:54.449204 14205 catalog_manager.cc:1259] Loaded cluster ID: 9b0fca54f3b649fba3b16596d02845cb
I20260811 00:23:54.449337 14205 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: loading cluster ID for follower catalog manager: success
I20260811 00:23:54.453672 14205 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab6383571e9743eaa6f2a038cb9649a4: acquiring CA information for follower catalog manager: success
F20260811 00:23:54.456338 14205 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f62e1ca99ad google::LogMessage::Fail() at ??:0
@ 0x7f62e1cac88a google::LogMessage::SendToLog() at ??:0
@ 0x7f62e1ca94de google::LogMessage::Flush() at ??:0
@ 0x7f62e1caaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f62e5a0d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f62f76e2bbf kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f62f76e217b kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f62f76d1247 kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f62f76cac56 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f62f776500c kudu::master::CatalogManagerBgTasks::Init()::$_0::operator()() at ??:0
@ 0x7f62f7764e89 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f62e48d3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f62e7bad6db start_thread at ??:0
@ 0x7f62e0b4361f clone at ??:0
W20260811 00:23:54.781328 14172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:54.781687 14172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:54.781826 14172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:54.789727 14172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:54.789870 14172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:54.789999 14172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:23:54.800684 14172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:46443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=39217
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:54.802205 14172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:54.803697 14172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:54.817626 14244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:54.817778 14253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:54.817821 14246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:54.818987 14172 server_base.cc:1034] running on GCE node
I20260811 00:23:54.819669 14172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:54.820775 14172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:54.822022 14172 hybrid_clock.cc:648] HybridClock initialized: now 1786407834821966 us; error 42 us; skew 500 ppm
I20260811 00:23:54.822415 14172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:54 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407834, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:54.826201 14172 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:23:54.828025 14172 webserver.cc:460] Webserver started at http://127.16.71.129:39217/ using document root <none> and password file <none>
I20260811 00:23:54.828914 14172 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:54.829073 14172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:54.835598 14172 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 00:23:54.838936 14266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:54.840852 14172 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:23:54.841064 14172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "9ae8a8cc6e8343e699a46c8c0daa41e9"
format_stamp: "Formatted at 2026-08-11 00:23:08 on dist-test-slave-f7th"
server_key: "8195b21e634b2e4ea61f3cc7aafef82a"
server_key_iv: "5928e605c368ca860dbed1c3df74df63"
server_key_version: "encryptionkey@0"
I20260811 00:23:54.842196 14172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:54.859484 14172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:54.864423 14172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:54.864771 14172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:54.865983 14172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:54.867586 14172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:54.867708 14172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:54.867832 14172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:54.867909 14172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:54.904102 14172 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:46443
I20260811 00:23:54.904153 14479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:46443 every 8 connection(s)
I20260811 00:23:54.905779 14172 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:23:54.908335 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 14172
I20260811 00:23:54.909058 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:37675
--local_ip_for_outbound_sockets=127.16.71.130
--tserver_master_addrs=127.16.71.190:45917
--webserver_port=37315
--webserver_interface=127.16.71.130
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:23:54.911794 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:54.909439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:46065)
0811 00:23:54.910442 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:54.910752 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":442,"threads_started":1}
W20260811 00:23:54.912689 14480 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:45917 (0 consecutive failures): Network error: Failed to ping master at 127.16.71.190:45917: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
I20260811 00:23:54.913667 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:54.913297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:51681)
0811 00:23:54.913462 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:54.913546 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 00:23:54.914620 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:54.914291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:34335)
0811 00:23:54.914415 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:54.914498 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 00:23:54.914865 14480 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:23:55.287719 14485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:55.288259 14485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:55.288445 14485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:55.297698 14485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:55.297858 14485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:55.298009 14485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:23:55.308802 14485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:37675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=37315
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:55.310312 14485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:55.311800 14485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:55.327121 14491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:55.327080 14490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:55.327148 14494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:55.328186 14485 server_base.cc:1034] running on GCE node
I20260811 00:23:55.328974 14485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:55.330106 14485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:55.331449 14485 hybrid_clock.cc:648] HybridClock initialized: now 1786407835331382 us; error 60 us; skew 500 ppm
I20260811 00:23:55.331848 14485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:55 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407835, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:55.335466 14485 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:23:55.337384 14485 webserver.cc:460] Webserver started at http://127.16.71.130:37315/ using document root <none> and password file <none>
I20260811 00:23:55.338199 14485 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:55.338318 14485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:55.344516 14485 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:23:55.347831 14505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:55.349509 14485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:23:55.349715 14485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "88e2276d75594bad8fbe5602365c55f6"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "6c1cbe4ecc78f1e7ac4b93d4899ea886"
server_key_iv: "f38b2495a9b8d6f3be46185c127ad318"
server_key_version: "encryptionkey@0"
I20260811 00:23:55.350818 14485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:55.366093 14485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:55.371173 14485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:55.371474 14485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:55.372562 14485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:55.374116 14485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:55.374238 14485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:55.374351 14485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:55.374421 14485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:55.407923 14485 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:37675
I20260811 00:23:55.407999 14704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:37675 every 8 connection(s)
I20260811 00:23:55.409574 14485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:23:55.414542 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 14485
I20260811 00:23:55.415162 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:55.412913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:47003)
0811 00:23:55.413739 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:55.414026 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":302,"threads_started":1}
I20260811 00:23:55.415217 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:36913
--local_ip_for_outbound_sockets=127.16.71.131
--tserver_master_addrs=127.16.71.190:45917
--webserver_port=44971
--webserver_interface=127.16.71.131
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:23:55.416157 14708 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:45917 (0 consecutive failures): Network error: Failed to ping master at 127.16.71.190:45917: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
I20260811 00:23:55.417531 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:55.416874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:38651)
0811 00:23:55.417188 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:55.417291 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 00:23:55.418705 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:55.418266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:37187)
0811 00:23:55.418421 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:55.418546 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 00:23:55.419031 14708 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:23:55.779757 14712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:23:55.780098 14712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:23:55.780217 14712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:23:55.787837 14712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:23:55.787979 14712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:23:55.788127 14712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:23:55.798421 14712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:36913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=44971
--tserver_master_addrs=127.16.71.190:45917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:23:55.799867 14712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:23:55.801308 14712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:23:55.815371 14719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:55.815408 14723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:23:55.815974 14718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:23:55.817394 14712 server_base.cc:1034] running on GCE node
I20260811 00:23:55.818156 14712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:23:55.819242 14712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:23:55.820502 14712 hybrid_clock.cc:648] HybridClock initialized: now 1786407835820431 us; error 54 us; skew 500 ppm
I20260811 00:23:55.820889 14712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:23:55 dist-test-slave-f7th krb5kdc[12118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407835, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:23:55.824501 14712 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:23:55.826268 14712 webserver.cc:460] Webserver started at http://127.16.71.131:44971/ using document root <none> and password file <none>
I20260811 00:23:55.827049 14712 fs_manager.cc:359] Metadata directory not provided
I20260811 00:23:55.827186 14712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:23:55.833796 14712 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:23:55.837284 14742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:55.839046 14712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:23:55.839264 14712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "d88c03b839b34bafb52dae8016f07be2"
format_stamp: "Formatted at 2026-08-11 00:23:09 on dist-test-slave-f7th"
server_key: "018023eef7b31ac8e6a331239e71b0bc"
server_key_iv: "98eaf1cc7ec5ccdcea75bbd33d5e2fe2"
server_key_version: "encryptionkey@0"
I20260811 00:23:55.840413 14712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:23:55.863335 14712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:23:55.868304 14712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:23:55.868598 14712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:23:55.869840 14712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:23:55.871448 14712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:23:55.871574 14712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:55.871706 14712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:23:55.871876 14712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:23:55.905840 14712 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:36913
I20260811 00:23:55.905936 14942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:36913 every 8 connection(s)
I20260811 00:23:55.907438 14712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:23:55.908980 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 14712
I20260811 00:23:55.913762 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:55.911638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:58123)
0811 00:23:55.912457 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:55.912764 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":312,"threads_started":1}
W20260811 00:23:55.914626 14944 heartbeater.cc:644] Failed to heartbeat to 127.16.71.190:45917 (0 consecutive failures): Network error: Failed to ping master at 127.16.71.190:45917: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
I20260811 00:23:55.915627 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:55.915246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:37555)
0811 00:23:55.915411 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:55.915496 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 00:23:55.916275 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:55.915444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:43583)
0811 00:23:55.915948 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:55.916054 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":254,"threads_started":1}
I20260811 00:23:55.916679 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:55.916252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:44495)
0811 00:23:55.916399 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:55.916503 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:23:55.916939 14944 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:23:56.421021 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:56.420068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:44363)
0811 00:23:56.420583 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:56.420743 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":249,"threads_started":1}
I20260811 00:23:56.918653 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:56.917507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:54659)
0811 00:23:56.918185 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:56.918323 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":292,"threads_started":1}
I20260811 00:23:56.918751 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:56.917544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:37845)
0811 00:23:56.918441 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:56.918556 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":630,"threads_started":1}
I20260811 00:23:57.422812 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:57.421996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:52231)
0811 00:23:57.422532 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:57.422630 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":253,"threads_started":1}
I20260811 00:23:57.920557 14961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:57.919724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:43083)
0811 00:23:57.920271 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:57.920363 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":312,"threads_started":1}
I20260811 00:23:57.920585 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:57.919754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:49869)
0811 00:23:57.920324 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:57.920407 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":321,"threads_started":1}
I20260811 00:23:58.424742 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:58.423852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:38757)
0811 00:23:58.424387 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:58.424488 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":274,"threads_started":1}
I20260811 00:23:58.922538 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:58.921677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:52059)
0811 00:23:58.922234 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:58.922336 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":287,"threads_started":1}
I20260811 00:23:58.922559 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:58.921691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:40619)
0811 00:23:58.922277 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:58.922371 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":337,"threads_started":1}
I20260811 00:23:59.426642 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:59.425810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:40753)
0811 00:23:59.426323 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:59.426423 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":247,"threads_started":1}
I20260811 00:23:59.924464 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:59.923624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:48303)
0811 00:23:59.924183 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:59.924278 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":313,"threads_started":1}
I20260811 00:23:59.924878 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:23:59.923627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:45341)
0811 00:23:59.924551 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 00:23:59.924681 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":689,"threads_started":1}
I20260811 00:24:00.428608 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:00.427811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:43983)
0811 00:24:00.428326 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:00.428408 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":279,"threads_started":1}
I20260811 00:24:00.926482 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:00.925516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:45683)
0811 00:24:00.926090 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:00.926238 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":271,"threads_started":1}
I20260811 00:24:00.926743 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:00.925819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:44015)
0811 00:24:00.926449 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:00.926533 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":369,"threads_started":1}
I20260811 00:24:01.430413 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:01.429623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:33791)
0811 00:24:01.430124 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:01.430233 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":276,"threads_started":1}
I20260811 00:24:01.929116 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:01.927888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:37877)
0811 00:24:01.928802 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:01.928923 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":431,"threads_started":1}
I20260811 00:24:01.929643 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:01.927846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:36525)
0811 00:24:01.929121 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:01.929224 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1136,"thread_start_us":906,"threads_started":1}
I20260811 00:24:02.432750 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:02.431626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:53709)
0811 00:24:02.432280 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:02.432394 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":301,"threads_started":1}
I20260811 00:24:02.931564 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:02.930329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:47801)
0811 00:24:02.931220 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:02.931321 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":520,"threads_started":1}
I20260811 00:24:02.932274 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:02.931288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:49789)
0811 00:24:02.932002 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:02.932106 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":314,"threads_started":1}
I20260811 00:24:03.434983 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:03.433979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:37139)
0811 00:24:03.434633 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:03.434781 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":279,"threads_started":1}
I20260811 00:24:03.933974 14998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:03.932874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:42503)
0811 00:24:03.933556 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:03.933700 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":353,"threads_started":1}
I20260811 00:24:03.934680 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:03.933450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:33933)
0811 00:24:03.934268 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:03.934400 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":311,"threads_started":1}
I20260811 00:24:04.436995 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:04.436114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:58541)
0811 00:24:04.436691 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:04.436801 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":258,"threads_started":1}
I20260811 00:24:04.936269 15012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:04.935429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:55517)
0811 00:24:04.935985 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:04.936085 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":280,"threads_started":1}
I20260811 00:24:04.936594 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:04.935717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:40627)
0811 00:24:04.936300 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:04.936414 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":364,"threads_started":1}
I20260811 00:24:05.439139 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:05.438202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:39371)
0811 00:24:05.438843 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:05.438938 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":383,"threads_started":1}
I20260811 00:24:05.938355 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:05.937458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:48041)
0811 00:24:05.938058 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:05.938175 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":301,"threads_started":1}
I20260811 00:24:05.938485 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:05.937713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:42879)
0811 00:24:05.938210 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:05.938306 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":272,"threads_started":1}
I20260811 00:24:06.441069 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:06.440222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:59483)
0811 00:24:06.440747 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:06.440858 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":229,"threads_started":1}
I20260811 00:24:06.940361 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:06.939481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:49401)
0811 00:24:06.940065 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:06.940168 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":277,"threads_started":1}
I20260811 00:24:06.940694 15023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:06.939488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:42511)
0811 00:24:06.940365 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:06.940457 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":649,"threads_started":1}
I20260811 00:24:07.443264 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:07.442317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:40711)
0811 00:24:07.442942 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:07.443060 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":304,"threads_started":1}
I20260811 00:24:07.942399 15028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:07.941513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:51223)
0811 00:24:07.942122 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:07.942224 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":300,"threads_started":1}
I20260811 00:24:07.942824 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:07.941808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:34583)
0811 00:24:07.942476 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:07.942598 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":400,"threads_started":1}
I20260811 00:24:08.445499 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:08.444511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:43037)
0811 00:24:08.445090 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:08.445221 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":263,"threads_started":1}
I20260811 00:24:08.944653 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:08.943733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:40005)
0811 00:24:08.944242 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:08.944339 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":263,"threads_started":1}
I20260811 00:24:08.944609 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:08.943588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:60505)
0811 00:24:08.944255 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:08.944400 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":354,"threads_started":1}
I20260811 00:24:09.447575 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:09.446716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:46307)
0811 00:24:09.447253 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:09.447361 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":284,"threads_started":1}
I20260811 00:24:09.947008 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:09.945984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:37249)
0811 00:24:09.946544 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:09.946673 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":259,"threads_started":1}
I20260811 00:24:09.947633 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:09.945954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:33747)
0811 00:24:09.947335 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:09.947450 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1244,"thread_start_us":1126,"threads_started":1}
I20260811 00:24:10.449649 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:10.448853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.130:45319)
0811 00:24:10.449345 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:10.449443 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":263,"threads_started":1}
I20260811 00:24:10.917160 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 14172
I20260811 00:24:10.942291 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:10.918495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55020 (local address 127.16.71.129:46443)
0811 00:24:10.919192 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:10.919212 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:10.919264 (+ 52us) server_negotiation.cc:407] Connection header received
0811 00:24:10.921119 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:10.921131 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:10.921692 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:10.922049 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:10.923711 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:10.925209 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:10.926650 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:10.927462 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:10.927627 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:10.934690 (+ 7063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:10.934703 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:10.934725 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:10.934781 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:10.938175 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:10.939161 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:10.939170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:10.939179 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:10.939415 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:10.939913 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:10.939917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:10.939919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:10.940567 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:10.940929 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:10.941463 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 00:24:10.941899 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":335,"threads_started":1}
I20260811 00:24:10.943866 14469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:10.949663 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:10.948789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.131:45211)
0811 00:24:10.949328 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:10.949465 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":266,"threads_started":1}
I20260811 00:24:11.032791 15046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:11.030826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45917 (local address 127.16.71.129:60875)
0811 00:24:11.032130 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:11.032397 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.190:45917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1154,"thread_start_us":956,"threads_started":1}
I20260811 00:24:11.033298 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 14172
I20260811 00:24:11.068536 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 14485
I20260811 00:24:11.087566 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:11.069552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.16.71.130:37675)
0811 00:24:11.070100 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:11.070114 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:11.070155 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:24:11.071848 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:11.071858 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:11.072332 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:11.072802 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:11.074382 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:11.075669 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:11.076999 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:11.077683 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:11.077900 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:11.080842 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:11.080854 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:11.080875 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:11.080935 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:11.083877 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:11.084707 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:11.084715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:11.084722 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:11.084920 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:11.085337 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:11.085341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:11.085344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:11.085930 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:11.086264 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:11.086748 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 00:24:11.087237 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":249,"threads_started":1}
I20260811 00:24:11.089229 14694 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:11.175423 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 14485
I20260811 00:24:11.206383 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 14712
I20260811 00:24:11.230173 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:11.207699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35130 (local address 127.16.71.131:36913)
0811 00:24:11.208568 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:11.208585 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:11.208685 (+ 100us) server_negotiation.cc:407] Connection header received
0811 00:24:11.210705 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:11.210720 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:11.211294 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:11.211680 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:11.213825 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:11.215348 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:11.217660 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:11.218596 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:11.218805 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:11.222500 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:11.222514 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:11.222535 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:11.222593 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:11.226135 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:11.227137 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:11.227146 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:11.227155 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:11.227382 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:11.227856 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:11.227861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:11.227863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:11.228542 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:11.228890 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:11.229322 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 00:24:11.229838 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":458,"threads_started":1}
I20260811 00:24:11.231786 14934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:11.322824 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 14712
2026-08-11T00:24:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65416 ms)
[----------] 30 tests from SecurityITest (323931 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:11 dist-test-slave-f7th krb5kdc[15057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:11 dist-test-slave-f7th krb5kdc[15057](info): set up 2 sockets
Aug 11 00:24:11 dist-test-slave-f7th krb5kdc[15057](info): commencing operation
krb5kdc: starting...
W20260811 00:24:12.647704 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:12 dist-test-slave-f7th krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:24:12.984650 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40961
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:40961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:13.382570 15088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:13.383005 15088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:13.383109 15088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:13.391240 15088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:13.391390 15088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:13.391438 15088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:13.391497 15088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:13.391542 15088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:13.402253 15088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:40961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:13.403638 15088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:13.405020 15088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:13.418578 15097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:13.418500 15100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:13.418538 15098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:13.418900 15088 server_base.cc:1034] running on GCE node
I20260811 00:24:13.420140 15088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:13.421890 15088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:13.423162 15088 hybrid_clock.cc:648] HybridClock initialized: now 1786407853423095 us; error 47 us; skew 500 ppm
I20260811 00:24:13.423635 15088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:13 dist-test-slave-f7th krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:13.427373 15088 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:24:13.429306 15088 webserver.cc:460] Webserver started at http://127.16.71.190:44805/ using document root <none> and password file <none>
I20260811 00:24:13.430086 15088 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:13.430226 15088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:13.430555 15088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:13.434077 15088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "e19ec85cf76042178440dd3805d183fb"
format_stamp: "Formatted at 2026-08-11 00:24:13 on dist-test-slave-f7th"
server_key: "34e54567d7486bed00e257171b7d1e37"
server_key_iv: "2d58c472a6dc1e0c509a1373c7d61db1"
server_key_version: "encryptionkey@0"
I20260811 00:24:13.435007 15088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "e19ec85cf76042178440dd3805d183fb"
format_stamp: "Formatted at 2026-08-11 00:24:13 on dist-test-slave-f7th"
server_key: "34e54567d7486bed00e257171b7d1e37"
server_key_iv: "2d58c472a6dc1e0c509a1373c7d61db1"
server_key_version: "encryptionkey@0"
I20260811 00:24:13.441442 15088 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:24:13.446249 15112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:13.448381 15088 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:24:13.448736 15088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "e19ec85cf76042178440dd3805d183fb"
format_stamp: "Formatted at 2026-08-11 00:24:13 on dist-test-slave-f7th"
server_key: "34e54567d7486bed00e257171b7d1e37"
server_key_iv: "2d58c472a6dc1e0c509a1373c7d61db1"
server_key_version: "encryptionkey@0"
I20260811 00:24:13.449016 15088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:13.496492 15088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:13.501278 15088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:13.501591 15088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:13.525853 15088 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:40961
I20260811 00:24:13.525877 15211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:40961 every 8 connection(s)
I20260811 00:24:13.528143 15088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:13.534679 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 15088
I20260811 00:24:13.534920 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:13.535915 16670 external_mini_cluster.cc:1428] Setting key 1ecf6f4dfd6241c72ac87d3d3157341d
I20260811 00:24:13.537149 15212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:24:13 dist-test-slave-f7th krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:13.555398 15212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb: Bootstrap starting.
I20260811 00:24:13.560011 15212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:13.561467 15212 log.cc:824] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:13.564992 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:13.538346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.16.71.190:40961)
0811 00:24:13.539377 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:13.539394 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:13.539463 (+ 69us) server_negotiation.cc:407] Connection header received
0811 00:24:13.541896 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:13.541936 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:13.542658 (+ 722us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:13.543191 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:13.544945 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:13.546898 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:13.548674 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:13.549828 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:13.550060 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:13.555815 (+ 5755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:13.555824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:13.555842 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:13.555888 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:13.559974 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:13.561187 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:13.561198 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:13.561207 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:13.561454 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:13.561906 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:13.561911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:13.561914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:13.562629 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:13.562935 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:13.563455 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 00:24:13.563920 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":412,"threads_started":1}
I20260811 00:24:13.565511 15212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb: No bootstrap required, opened a new log
I20260811 00:24:13.576467 15212 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19ec85cf76042178440dd3805d183fb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40961 } }
I20260811 00:24:13.576946 15212 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:13.577069 15212 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19ec85cf76042178440dd3805d183fb, State: Initialized, Role: FOLLOWER
I20260811 00:24:13.577701 15212 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19ec85cf76042178440dd3805d183fb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40961 } }
I20260811 00:24:13.577934 15212 raft_consensus.cc:388] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:13.578081 15212 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:13.578265 15212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:13.580572 15212 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19ec85cf76042178440dd3805d183fb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40961 } }
I20260811 00:24:13.581249 15212 leader_election.cc:302] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e19ec85cf76042178440dd3805d183fb; no voters:
I20260811 00:24:13.581887 15212 leader_election.cc:288] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:13.582075 15223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:13.582831 15223 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [term 1 LEADER]: Becoming Leader. State: Replica: e19ec85cf76042178440dd3805d183fb, State: Running, Role: LEADER
I20260811 00:24:13.583472 15223 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19ec85cf76042178440dd3805d183fb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40961 } }
I20260811 00:24:13.584070 15212 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:13.586587 15225 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19ec85cf76042178440dd3805d183fb. Latest consensus state: current_term: 1 leader_uuid: "e19ec85cf76042178440dd3805d183fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19ec85cf76042178440dd3805d183fb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40961 } } }
I20260811 00:24:13.586956 15225 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:13.587870 15234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:13.587728 15224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e19ec85cf76042178440dd3805d183fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19ec85cf76042178440dd3805d183fb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40961 } } }
I20260811 00:24:13.588150 15224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:13.593883 15234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:13.603986 15234 catalog_manager.cc:1347] Generated new cluster ID: 2765905c016f4b3d951aa0991118020b
I20260811 00:24:13.604118 15234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:13.625859 15234 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:13.627393 15234 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:13.636938 15234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb: Generated new TSK 0
I20260811 00:24:13.638075 15234 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:24:13.734863 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40961
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:14.132987 15261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:14.133297 15261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:14.133396 15261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:14.141384 15261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:14.141538 15261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:14.141659 15261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:24:14.152101 15261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:14.153645 15261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:14.155277 15261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:14.170429 15271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:14.170547 15273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:14.170666 15276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:14.171932 15261 server_base.cc:1034] running on GCE node
I20260811 00:24:14.172819 15261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:14.173967 15261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:14.175256 15261 hybrid_clock.cc:648] HybridClock initialized: now 1786407854175185 us; error 50 us; skew 500 ppm
I20260811 00:24:14.175725 15261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:14 dist-test-slave-f7th krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407854, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:14.179572 15261 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:24:14.181483 15261 webserver.cc:460] Webserver started at http://127.16.71.129:42275/ using document root <none> and password file <none>
I20260811 00:24:14.182312 15261 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:14.182449 15261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:14.182821 15261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:14.186352 15261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "958dd75ea2984968a38930a1a93f647f"
format_stamp: "Formatted at 2026-08-11 00:24:14 on dist-test-slave-f7th"
server_key: "a1c3c8cb829bece411767a514bbbc295"
server_key_iv: "7fffbdd8a1337e419d85fd2e9fb8ac83"
server_key_version: "encryptionkey@0"
I20260811 00:24:14.187258 15261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "958dd75ea2984968a38930a1a93f647f"
format_stamp: "Formatted at 2026-08-11 00:24:14 on dist-test-slave-f7th"
server_key: "a1c3c8cb829bece411767a514bbbc295"
server_key_iv: "7fffbdd8a1337e419d85fd2e9fb8ac83"
server_key_version: "encryptionkey@0"
I20260811 00:24:14.193465 15261 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:24:14.197562 15288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:14.199098 15261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:24:14.199278 15261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "958dd75ea2984968a38930a1a93f647f"
format_stamp: "Formatted at 2026-08-11 00:24:14 on dist-test-slave-f7th"
server_key: "a1c3c8cb829bece411767a514bbbc295"
server_key_iv: "7fffbdd8a1337e419d85fd2e9fb8ac83"
server_key_version: "encryptionkey@0"
I20260811 00:24:14.199524 15261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:14.215830 15261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:14.220556 15261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:14.220923 15261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:14.222031 15261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:14.223589 15261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:14.223670 15261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:14.223799 15261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:14.223867 15261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:14.257866 15261 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:40963
I20260811 00:24:14.257915 15478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:40963 every 8 connection(s)
I20260811 00:24:14.259274 15261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:24:14.265480 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 15261
I20260811 00:24:14.265710 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:24:14.266516 16670 external_mini_cluster.cc:1428] Setting key 8be9e2e1a8b1c6ce3b5c507b6191e8bf
Aug 11 00:24:14 dist-test-slave-f7th krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407854, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:14.282531 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:14.262020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:34997 (local address 127.16.71.190:40961)
0811 00:24:14.262505 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:14.262510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:14.263655 (+ 1145us) server_negotiation.cc:407] Connection header received
0811 00:24:14.265712 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:14.265719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:14.265826 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:14.265962 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:14.267819 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.268840 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.270497 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.271114 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.271253 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:14.275329 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:14.275336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:14.275341 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:14.275385 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:14.278571 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:14.279696 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:14.279701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:14.279704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:14.279864 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:14.280467 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:14.280472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:14.280475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:14.280840 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:14.280976 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:14.281926 (+ 950us) server_negotiation.cc:299] Negotiation successful
0811 00:24:14.282169 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":249,"threads_started":1}
I20260811 00:24:14.284029 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:14.262645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40961 (local address 127.16.71.129:34997)
0811 00:24:14.263452 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:14.263502 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:14.265347 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:14.266196 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:14.266206 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:14.266866 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:14.267641 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:14.267664 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.269014 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.269018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:14.270315 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:14.270331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.271359 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.271364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:14.271613 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:14.272293 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:14.272318 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:14.274937 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:14.278792 (+ 3855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:14.278805 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:14.278829 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:14.279552 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:14.280026 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:14.280031 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:14.280034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:14.280355 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:14.280988 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:14.280997 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:14.281513 (+ 516us) client_negotiation.cc:769] Sending connection context
0811 00:24:14.281847 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 00:24:14.282480 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":304,"threads_started":1}
I20260811 00:24:14.285823 15479 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40961
I20260811 00:24:14.286214 15479 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:14.287246 15479 heartbeater.cc:511] Master 127.16.71.190:40961 requested a full tablet report, sending...
I20260811 00:24:14.289985 15145 ts_manager.cc:194] Registered new tserver with Master: 958dd75ea2984968a38930a1a93f647f (127.16.71.129:40963)
I20260811 00:24:14.291723 15145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:34997
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:24:14.342271 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40961
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:14.708942 15489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:14.709270 15489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:14.709367 15489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:14.717154 15489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:14.717290 15489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:14.717417 15489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:24:14.727816 15489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:14.729336 15489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:14.730751 15489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:14.744727 15499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:14.744675 15497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:14.745097 15496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:14.746353 15489 server_base.cc:1034] running on GCE node
I20260811 00:24:14.747036 15489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:14.748039 15489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:14.749274 15489 hybrid_clock.cc:648] HybridClock initialized: now 1786407854749213 us; error 44 us; skew 500 ppm
I20260811 00:24:14.749635 15489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:14 dist-test-slave-f7th krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407854, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:14.753208 15489 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:24:14.755046 15489 webserver.cc:460] Webserver started at http://127.16.71.130:46129/ using document root <none> and password file <none>
I20260811 00:24:14.755823 15489 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:14.755959 15489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:14.756297 15489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:14.759563 15489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "ea66d4eaca3940a4bf1cbcb04460541d"
format_stamp: "Formatted at 2026-08-11 00:24:14 on dist-test-slave-f7th"
server_key: "7a4608fea3f8b499b131689946873cfd"
server_key_iv: "783acefcbfc12b8a4c8c5d18f8e4a762"
server_key_version: "encryptionkey@0"
I20260811 00:24:14.760402 15489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "ea66d4eaca3940a4bf1cbcb04460541d"
format_stamp: "Formatted at 2026-08-11 00:24:14 on dist-test-slave-f7th"
server_key: "7a4608fea3f8b499b131689946873cfd"
server_key_iv: "783acefcbfc12b8a4c8c5d18f8e4a762"
server_key_version: "encryptionkey@0"
I20260811 00:24:14.766638 15489 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:14.770740 15513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:14.772300 15489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:24:14.772508 15489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "ea66d4eaca3940a4bf1cbcb04460541d"
format_stamp: "Formatted at 2026-08-11 00:24:14 on dist-test-slave-f7th"
server_key: "7a4608fea3f8b499b131689946873cfd"
server_key_iv: "783acefcbfc12b8a4c8c5d18f8e4a762"
server_key_version: "encryptionkey@0"
I20260811 00:24:14.772806 15489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:14.788820 15489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:14.793853 15489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:14.794183 15489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:14.795236 15489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:14.796715 15489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:14.796797 15489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:14.796900 15489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:14.796958 15489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:14.829843 15489 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:44887
I20260811 00:24:14.829892 15721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:44887 every 8 connection(s)
I20260811 00:24:14.831310 15489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:24:14.835480 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 15489
I20260811 00:24:14.835677 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:24:14.836210 16670 external_mini_cluster.cc:1428] Setting key 506c22d489d29eb39b1b42b36cad16d7
Aug 11 00:24:14 dist-test-slave-f7th krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407854, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:14.855396 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:14.834228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:37845 (local address 127.16.71.190:40961)
0811 00:24:14.834736 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:14.834741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:14.835941 (+ 1200us) server_negotiation.cc:407] Connection header received
0811 00:24:14.838112 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:14.838117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:14.838190 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:14.838279 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:14.840041 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.841139 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.842936 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.843539 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.843679 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:14.847975 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:14.847982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:14.847987 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:14.848035 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:14.851202 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:14.852319 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:14.852325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:14.852327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:14.852482 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:14.853152 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:14.853158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:14.853160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:14.853638 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:14.853780 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:14.854871 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0811 00:24:14.855101 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":258,"threads_started":1}
I20260811 00:24:14.856920 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:14.834843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40961 (local address 127.16.71.130:37845)
0811 00:24:14.835741 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:14.835791 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:14.837820 (+ 2029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:14.838576 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:14.838586 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:14.839153 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:14.839801 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:14.839821 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.841347 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.841351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:14.842736 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:14.842753 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:14.843777 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:14.843782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:14.844033 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:14.844799 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:14.844823 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:14.847656 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:14.851429 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:14.851440 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:14.851457 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:14.852112 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:14.852655 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:14.852660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:14.852664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:14.852976 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:14.853821 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:14.853830 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:14.854327 (+ 497us) client_negotiation.cc:769] Sending connection context
0811 00:24:14.854755 (+ 428us) client_negotiation.cc:240] Negotiation successful
0811 00:24:14.855347 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":358,"threads_started":1}
I20260811 00:24:14.858637 15722 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40961
I20260811 00:24:14.859025 15722 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:14.859902 15722 heartbeater.cc:511] Master 127.16.71.190:40961 requested a full tablet report, sending...
I20260811 00:24:14.861704 15145 ts_manager.cc:194] Registered new tserver with Master: ea66d4eaca3940a4bf1cbcb04460541d (127.16.71.130:44887)
I20260811 00:24:14.862594 15145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:37845
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:24:14.912415 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40961
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:15.284209 15734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:15.284579 15734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:15.284750 15734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:15.292644 15734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:15.292804 15734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:15.292970 15734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:24:15.295835 15479 heartbeater.cc:503] Master 127.16.71.190:40961 was elected leader, sending a full tablet report...
I20260811 00:24:15.304190 15734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:40961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:15.305722 15734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:15.307258 15734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:15.321369 15744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:15.321306 15750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:15.321349 15743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:15.322383 15734 server_base.cc:1034] running on GCE node
I20260811 00:24:15.323072 15734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:15.324152 15734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:15.325450 15734 hybrid_clock.cc:648] HybridClock initialized: now 1786407855325379 us; error 75 us; skew 500 ppm
I20260811 00:24:15.325821 15734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:15 dist-test-slave-f7th krb5kdc[15057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407855, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:15.329447 15734 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:24:15.331303 15734 webserver.cc:460] Webserver started at http://127.16.71.131:39155/ using document root <none> and password file <none>
I20260811 00:24:15.332098 15734 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:15.332255 15734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:15.332640 15734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:15.335965 15734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807"
format_stamp: "Formatted at 2026-08-11 00:24:15 on dist-test-slave-f7th"
server_key: "bca43b779f665d2d6f0b2c9958b04d34"
server_key_iv: "929551e11db1662fece89407d9f4f6af"
server_key_version: "encryptionkey@0"
I20260811 00:24:15.336889 15734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807"
format_stamp: "Formatted at 2026-08-11 00:24:15 on dist-test-slave-f7th"
server_key: "bca43b779f665d2d6f0b2c9958b04d34"
server_key_iv: "929551e11db1662fece89407d9f4f6af"
server_key_version: "encryptionkey@0"
I20260811 00:24:15.343078 15734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:15.347191 15763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:15.348839 15734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:24:15.349025 15734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807"
format_stamp: "Formatted at 2026-08-11 00:24:15 on dist-test-slave-f7th"
server_key: "bca43b779f665d2d6f0b2c9958b04d34"
server_key_iv: "929551e11db1662fece89407d9f4f6af"
server_key_version: "encryptionkey@0"
I20260811 00:24:15.349251 15734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:15.377678 15734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:15.382946 15734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:15.383288 15734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:15.384421 15734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:15.386237 15734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:15.386328 15734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:15.386459 15734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:15.386514 15734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:15.423273 15734 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:36455
I20260811 00:24:15.423319 15951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:36455 every 8 connection(s)
I20260811 00:24:15.424808 15734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:24:15.426721 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 15734
I20260811 00:24:15.426947 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:24:15.427489 16670 external_mini_cluster.cc:1428] Setting key 968e115db54c7707452106b3729a671e
Aug 11 00:24:15 dist-test-slave-f7th krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407855, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:15.449385 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.427937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:36281 (local address 127.16.71.190:40961)
0811 00:24:15.428457 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.428463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.432115 (+ 3652us) server_negotiation.cc:407] Connection header received
0811 00:24:15.432176 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.432179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.432268 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.432355 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:15.434305 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.435124 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.436936 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.437391 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.437535 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.441575 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:15.441580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:15.441583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:15.441624 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:15.445026 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.446474 (+ 1448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.446479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.446482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.446651 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.447241 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.447245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.447247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.447581 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:15.447749 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.448673 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.448968 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":250,"threads_started":1}
I20260811 00:24:15.450436 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.428594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:40961 (local address 127.16.71.131:36281)
0811 00:24:15.429653 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:15.429715 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:15.431756 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:15.432712 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:15.432728 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:15.433372 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:15.434078 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.434103 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.435302 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.435306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.436733 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.436759 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.437633 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.437638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.437865 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.438617 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:15.438643 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:15.441265 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:15.445233 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:15.445243 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:15.445262 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:15.446253 (+ 991us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:15.446783 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:15.446788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:15.446790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:15.447087 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:15.447729 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:15.447739 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:15.448166 (+ 427us) client_negotiation.cc:769] Sending connection context
0811 00:24:15.448448 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 00:24:15.449129 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":366,"threads_started":1}
I20260811 00:24:15.452251 15952 heartbeater.cc:347] Connected to a master server at 127.16.71.190:40961
I20260811 00:24:15.452697 15952 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:15.453715 15952 heartbeater.cc:511] Master 127.16.71.190:40961 requested a full tablet report, sending...
I20260811 00:24:15.455696 15143 ts_manager.cc:194] Registered new tserver with Master: d5b2549d4c4e4069ae4cc3d6eb9f6807 (127.16.71.131:36455)
I20260811 00:24:15.456838 15143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:36281
I20260811 00:24:15.457067 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:24:15.476466 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.462415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.16.71.190:40961)
0811 00:24:15.462589 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.462593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.463084 (+ 491us) server_negotiation.cc:407] Connection header received
0811 00:24:15.463225 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.463230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.463323 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.463455 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:15.465184 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.466022 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.467587 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.468059 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.468176 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.470763 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:15.470768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:15.470772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:15.470848 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:15.473594 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.474451 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.474455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.474458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.474601 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.474987 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.474991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.474993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.475373 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:15.475581 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.475887 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.476115 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:24:15.481385 15143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:24:15.488607 15143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:24:15.527915 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.502455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:40963 (local address 127.0.0.1:34682)
0811 00:24:15.514446 (+ 11991us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:15.516193 (+ 1747us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:15.516442 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:15.518532 (+ 2090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:15.518541 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:15.518579 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:15.519061 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.519077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.521211 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.521217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.524292 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.524317 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.526644 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.526650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.526916 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.527006 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 00:24:15.527156 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 00:24:15.527402 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11817,"mutex_wait_us":161,"thread_start_us":11633,"threads_started":1}
I20260811 00:24:15.527930 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.503098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38470 (local address 127.16.71.131:36455)
0811 00:24:15.518491 (+ 15393us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.518501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.518541 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:24:15.518650 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.518657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.518914 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.519109 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:15.519833 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.521438 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.524552 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.526172 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.526316 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.526549 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.527351 (+ 802us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.527559 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14844,"thread_start_us":15124,"threads_started":1}
I20260811 00:24:15.529101 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.502217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.16.71.130:44887)
0811 00:24:15.518866 (+ 16649us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.518874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.518904 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:24:15.519001 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.519009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.519253 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.519451 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:15.520105 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.521325 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.523991 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.525476 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.525654 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.525886 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.528295 (+ 2409us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.528488 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16487,"thread_start_us":16367,"threads_started":1}
I20260811 00:24:15.529142 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.503098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.16.71.129:40963)
0811 00:24:15.513464 (+ 10366us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.513476 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.516608 (+ 3132us) server_negotiation.cc:407] Connection header received
0811 00:24:15.516788 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.516797 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.517908 (+ 1111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.518157 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:15.519288 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.521019 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.524530 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.526075 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.526233 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.526487 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.528450 (+ 1963us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.528650 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10110,"thread_start_us":10385,"threads_started":1}
I20260811 00:24:15.528755 15978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.504652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36455 (local address 127.0.0.1:38470)
0811 00:24:15.517106 (+ 12454us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:15.517132 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:15.517291 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:15.519151 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:15.519156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:15.519184 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:15.519590 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.519608 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.521653 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.521663 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.524313 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.524324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.526714 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.526721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.527020 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.527061 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:24:15.527167 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 00:24:15.528317 (+ 1150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12264,"mutex_wait_us":1086,"thread_start_us":5847,"threads_started":1}
I20260811 00:24:15.529206 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.502161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:44887 (local address 127.0.0.1:54394)
0811 00:24:15.511090 (+ 8929us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:15.511152 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:15.511440 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:15.519473 (+ 8033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:15.519481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:15.519506 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:15.519934 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.519944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.521524 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.521530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.523744 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.523766 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.527666 (+ 3900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.527672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.527928 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.527962 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:24:15.528051 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:24:15.528854 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8533,"mutex_wait_us":660,"thread_start_us":15220,"threads_started":1}
I20260811 00:24:15.534250 15616 tablet_service.cc:1467] Processing CreateTablet for tablet 7c744b5d4ed541a1bfbc5af6d304c938 (DEFAULT_TABLE table=test-table [id=85efc7714df64f2eb4580e0762fb35f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:15.535722 15616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c744b5d4ed541a1bfbc5af6d304c938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:15.539220 15377 tablet_service.cc:1467] Processing CreateTablet for tablet 7c744b5d4ed541a1bfbc5af6d304c938 (DEFAULT_TABLE table=test-table [id=85efc7714df64f2eb4580e0762fb35f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:15.541219 15377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c744b5d4ed541a1bfbc5af6d304c938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:15.541040 15851 tablet_service.cc:1467] Processing CreateTablet for tablet 7c744b5d4ed541a1bfbc5af6d304c938 (DEFAULT_TABLE table=test-table [id=85efc7714df64f2eb4580e0762fb35f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:15.545187 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c744b5d4ed541a1bfbc5af6d304c938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:15.553103 15991 tablet_bootstrap.cc:492] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Bootstrap starting.
I20260811 00:24:15.556805 15992 tablet_bootstrap.cc:492] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Bootstrap starting.
I20260811 00:24:15.557853 15991 tablet_bootstrap.cc:654] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:15.559270 15991 log.cc:824] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:15.561089 15992 tablet_bootstrap.cc:654] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:15.562714 15992 log.cc:824] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:15.562999 15996 tablet_bootstrap.cc:492] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Bootstrap starting.
I20260811 00:24:15.566999 15996 tablet_bootstrap.cc:654] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:15.567286 15991 tablet_bootstrap.cc:492] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: No bootstrap required, opened a new log
I20260811 00:24:15.567291 15992 tablet_bootstrap.cc:492] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: No bootstrap required, opened a new log
I20260811 00:24:15.567620 15992 ts_tablet_manager.cc:1397] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 00:24:15.567619 15991 ts_tablet_manager.cc:1397] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260811 00:24:15.568313 15996 log.cc:824] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:15.571429 15996 tablet_bootstrap.cc:492] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: No bootstrap required, opened a new log
I20260811 00:24:15.571878 15996 ts_tablet_manager.cc:1397] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 00:24:15.579125 15991 raft_consensus.cc:348] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.579124 15992 raft_consensus.cc:348] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.579738 15992 raft_consensus.cc:374] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:15.579787 15991 raft_consensus.cc:374] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:15.579895 15991 raft_consensus.cc:729] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea66d4eaca3940a4bf1cbcb04460541d, State: Initialized, Role: FOLLOWER
I20260811 00:24:15.579988 15992 raft_consensus.cc:729] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 958dd75ea2984968a38930a1a93f647f, State: Initialized, Role: FOLLOWER
I20260811 00:24:15.580725 15991 consensus_queue.cc:260] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.580732 15992 consensus_queue.cc:260] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.581768 15996 raft_consensus.cc:348] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.582409 15996 raft_consensus.cc:374] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:15.582670 15722 heartbeater.cc:503] Master 127.16.71.190:40961 was elected leader, sending a full tablet report...
I20260811 00:24:15.582580 15996 raft_consensus.cc:729] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b2549d4c4e4069ae4cc3d6eb9f6807, State: Initialized, Role: FOLLOWER
I20260811 00:24:15.583005 15992 ts_tablet_manager.cc:1428] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260811 00:24:15.583284 15991 ts_tablet_manager.cc:1428] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Time spent starting tablet: real 0.016s user 0.013s sys 0.005s
W20260811 00:24:15.583612 15725 tablet.cc:2369] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:15.583793 15996 consensus_queue.cc:260] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.585541 15952 heartbeater.cc:503] Master 127.16.71.190:40961 was elected leader, sending a full tablet report...
I20260811 00:24:15.588207 15996 ts_tablet_manager.cc:1428] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Time spent starting tablet: real 0.016s user 0.003s sys 0.009s
I20260811 00:24:15.589444 16004 raft_consensus.cc:482] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:24:15.589860 16004 raft_consensus.cc:504] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.592021 16004 leader_election.cc:288] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 958dd75ea2984968a38930a1a93f647f (127.16.71.129:40963), d5b2549d4c4e4069ae4cc3d6eb9f6807 (127.16.71.131:36455)
I20260811 00:24:15.599349 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.592767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:37123 (local address 127.16.71.131:36455)
0811 00:24:15.592970 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.592977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.593286 (+ 309us) server_negotiation.cc:407] Connection header received
0811 00:24:15.593582 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.593587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.593653 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.593729 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:15.594408 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.595185 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.597466 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.598285 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.598437 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.598537 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.598954 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.599100 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:24:15.599363 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.593036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:38341 (local address 127.16.71.129:40963)
0811 00:24:15.593266 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.593271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.593396 (+ 125us) server_negotiation.cc:407] Connection header received
0811 00:24:15.593613 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.593617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.593681 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.593778 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:15.594428 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.595187 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.597480 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.598336 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.598542 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.598621 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.598984 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.599121 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:24:15.599555 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.592308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:36455 (local address 127.16.71.130:37123)
0811 00:24:15.593162 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:15.593185 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:15.593361 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:15.593770 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:15.593775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:15.593801 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:15.594248 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.594260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.595340 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.595346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.597274 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.597292 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.598565 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.598574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.598771 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.598810 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:24:15.598894 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 00:24:15.599053 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":77,"thread_start_us":375,"threads_started":1}
I20260811 00:24:15.599601 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.592336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:40963 (local address 127.16.71.130:38341)
0811 00:24:15.593298 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:15.593316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:15.593460 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:15.593813 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:15.593817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:15.593843 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:15.594264 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.594274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.595346 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.595352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.597270 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:15.597292 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.598604 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.598609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:15.598813 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.598839 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 00:24:15.598931 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 00:24:15.599080 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":93,"thread_start_us":940,"threads_started":1}
I20260811 00:24:15.600797 15416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c744b5d4ed541a1bfbc5af6d304c938" candidate_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958dd75ea2984968a38930a1a93f647f" is_pre_election: true
I20260811 00:24:15.600798 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c744b5d4ed541a1bfbc5af6d304c938" candidate_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" is_pre_election: true
I20260811 00:24:15.601338 15875 raft_consensus.cc:2454] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea66d4eaca3940a4bf1cbcb04460541d in term 0.
I20260811 00:24:15.601354 15416 raft_consensus.cc:2454] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea66d4eaca3940a4bf1cbcb04460541d in term 0.
I20260811 00:24:15.602108 15523 leader_election.cc:302] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5b2549d4c4e4069ae4cc3d6eb9f6807, ea66d4eaca3940a4bf1cbcb04460541d; no voters:
I20260811 00:24:15.602569 16004 raft_consensus.cc:2781] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:24:15.602730 16004 raft_consensus.cc:482] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:24:15.602844 16004 raft_consensus.cc:3037] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:15.605486 16004 raft_consensus.cc:504] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.606158 16004 leader_election.cc:288] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [CANDIDATE]: Term 1 election: Requested vote from peers 958dd75ea2984968a38930a1a93f647f (127.16.71.129:40963), d5b2549d4c4e4069ae4cc3d6eb9f6807 (127.16.71.131:36455)
I20260811 00:24:15.606705 15416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c744b5d4ed541a1bfbc5af6d304c938" candidate_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958dd75ea2984968a38930a1a93f647f"
I20260811 00:24:15.606773 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c744b5d4ed541a1bfbc5af6d304c938" candidate_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807"
I20260811 00:24:15.606940 15416 raft_consensus.cc:3037] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:15.606992 15875 raft_consensus.cc:3037] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:15.609330 15875 raft_consensus.cc:2454] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea66d4eaca3940a4bf1cbcb04460541d in term 1.
I20260811 00:24:15.609334 15416 raft_consensus.cc:2454] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea66d4eaca3940a4bf1cbcb04460541d in term 1.
I20260811 00:24:15.609797 15523 leader_election.cc:302] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5b2549d4c4e4069ae4cc3d6eb9f6807, ea66d4eaca3940a4bf1cbcb04460541d; no voters:
I20260811 00:24:15.610155 16004 raft_consensus.cc:2781] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:15.610625 16004 raft_consensus.cc:686] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 1 LEADER]: Becoming Leader. State: Replica: ea66d4eaca3940a4bf1cbcb04460541d, State: Running, Role: LEADER
I20260811 00:24:15.611121 16004 consensus_queue.cc:237] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } }
I20260811 00:24:15.616400 15143 catalog_manager.cc:5495] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d reported cstate change: term changed from 0 to 1, leader changed from <none> to ea66d4eaca3940a4bf1cbcb04460541d (127.16.71.130). New cstate: current_term: 1 leader_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66d4eaca3940a4bf1cbcb04460541d" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 44887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 } health_report { overall_health: UNKNOWN } } }
I20260811 00:24:15.636404 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.630525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.16.71.130:44887)
0811 00:24:15.630752 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.630758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.630782 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:24:15.631065 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.631070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.631155 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.631298 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:24:15.631798 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.632548 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.634089 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.634512 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.634629 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.635202 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:24:15.635754 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:24:15.635904 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.636011 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.636107 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:24:15.645443 15875 raft_consensus.cc:1261] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 1 FOLLOWER]: Refusing update from remote peer ea66d4eaca3940a4bf1cbcb04460541d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:15.645468 15416 raft_consensus.cc:1261] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 1 FOLLOWER]: Refusing update from remote peer ea66d4eaca3940a4bf1cbcb04460541d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:15.646575 16013 consensus_queue.cc:1035] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5b2549d4c4e4069ae4cc3d6eb9f6807" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:15.647015 16004 consensus_queue.cc:1035] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [LEADER]: Connected to new peer: Peer: permanent_uuid: "958dd75ea2984968a38930a1a93f647f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:15.678205 15143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:15.678516 15143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:15.682530 15143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e19ec85cf76042178440dd3805d183fb: Sending DeleteTablet for 3 replicas of tablet 7c744b5d4ed541a1bfbc5af6d304c938
I20260811 00:24:15.684026 15616 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c744b5d4ed541a1bfbc5af6d304c938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:15 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:54394
I20260811 00:24:15.684003 15377 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c744b5d4ed541a1bfbc5af6d304c938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:15 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:34682
I20260811 00:24:15.684108 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c744b5d4ed541a1bfbc5af6d304c938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:15 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:38470
I20260811 00:24:15.686673 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 15261
I20260811 00:24:15.697045 16033 tablet_replica.cc:330] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: stopping tablet replica
I20260811 00:24:15.697854 16033 raft_consensus.cc:2229] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:24:15.698745 16033 raft_consensus.cc:2258] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:15.699796 16037 tablet_replica.cc:330] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: stopping tablet replica
I20260811 00:24:15.700387 16037 raft_consensus.cc:2229] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:15.700991 16038 tablet_replica.cc:330] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: stopping tablet replica
I20260811 00:24:15.701185 16037 raft_consensus.cc:2258] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:15.701442 16038 raft_consensus.cc:2229] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:15.701998 16038 raft_consensus.cc:2258] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:15.702558 16033 ts_tablet_manager.cc:1905] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:15.703958 16038 ts_tablet_manager.cc:1905] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:15.703958 16037 ts_tablet_manager.cc:1905] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:15.711920 16037 ts_tablet_manager.cc:1918] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:15.711913 16038 ts_tablet_manager.cc:1918] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:15.712146 16038 log.cc:1196] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/7c744b5d4ed541a1bfbc5af6d304c938
I20260811 00:24:15.712327 16037 log.cc:1196] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/7c744b5d4ed541a1bfbc5af6d304c938
I20260811 00:24:15.712543 16033 ts_tablet_manager.cc:1918] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:15.712792 16038 ts_tablet_manager.cc:1939] T 7c744b5d4ed541a1bfbc5af6d304c938 P d5b2549d4c4e4069ae4cc3d6eb9f6807: Deleting consensus metadata
I20260811 00:24:15.712816 16033 log.cc:1196] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/7c744b5d4ed541a1bfbc5af6d304c938
I20260811 00:24:15.713109 16037 ts_tablet_manager.cc:1939] T 7c744b5d4ed541a1bfbc5af6d304c938 P 958dd75ea2984968a38930a1a93f647f: Deleting consensus metadata
I20260811 00:24:15.713657 16033 ts_tablet_manager.cc:1939] T 7c744b5d4ed541a1bfbc5af6d304c938 P ea66d4eaca3940a4bf1cbcb04460541d: Deleting consensus metadata
I20260811 00:24:15.715075 15118 catalog_manager.cc:4841] TS d5b2549d4c4e4069ae4cc3d6eb9f6807 (127.16.71.131:36455): tablet 7c744b5d4ed541a1bfbc5af6d304c938 (table test-table [id=85efc7714df64f2eb4580e0762fb35f2]) successfully deleted
I20260811 00:24:15.715153 15115 catalog_manager.cc:4841] TS 958dd75ea2984968a38930a1a93f647f (127.16.71.129:40963): tablet 7c744b5d4ed541a1bfbc5af6d304c938 (table test-table [id=85efc7714df64f2eb4580e0762fb35f2]) successfully deleted
I20260811 00:24:15.716562 15116 catalog_manager.cc:4841] TS ea66d4eaca3940a4bf1cbcb04460541d (127.16.71.130:44887): tablet 7c744b5d4ed541a1bfbc5af6d304c938 (table test-table [id=85efc7714df64f2eb4580e0762fb35f2]) successfully deleted
Aug 11 00:24:15 dist-test-slave-f7th krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:24:15.730875 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.688492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.16.71.129:40963)
0811 00:24:15.688741 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.688747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.700205 (+ 11458us) server_negotiation.cc:407] Connection header received
0811 00:24:15.700451 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.700457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.700568 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.700724 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:15.716298 (+ 15574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.717227 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.719011 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.719439 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.719607 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.724328 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:15.724346 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:15.724364 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:15.724418 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:15.727624 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.728479 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.728488 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.728490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.728745 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.729159 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.729163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.729165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.729576 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:15.729869 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.730248 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.730487 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:24:15.731594 15474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:15.856017 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 15261
I20260811 00:24:15.893976 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 15489
Aug 11 00:24:15 dist-test-slave-f7th krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:24:15.911875 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:15.894895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.16.71.130:44887)
0811 00:24:15.895120 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:15.895127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:15.895150 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:24:15.895258 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:15.895263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:15.895356 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:15.895490 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:15.897226 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.898202 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.899683 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:15.900201 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:15.900336 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:15.905297 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:15.905313 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:15.905334 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:15.905385 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:15.908489 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.909519 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.909530 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.909533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.909711 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:15.910161 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:15.910166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:15.910168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:15.910591 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:15.910962 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:15.911265 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 00:24:15.911508 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:24:15.912973 15714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:16.033032 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 15489
I20260811 00:24:16.062673 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 15734
Aug 11 00:24:16 dist-test-slave-f7th krb5kdc[15057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:24:16.082774 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:16.063447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.16.71.131:36455)
0811 00:24:16.063622 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:16.063627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:16.063647 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:24:16.063782 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:16.063786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:16.063851 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:16.063980 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:16.065624 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:16.066406 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:16.067861 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:16.068312 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:16.068417 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:16.076808 (+ 8391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:16.076828 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:16.076846 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:16.076893 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:16.079811 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:16.080696 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:16.080706 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:16.080708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:16.080870 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:16.081368 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:16.081372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:16.081378 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:16.081785 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:16.082042 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:16.082314 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 00:24:16.082507 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:24:16.083441 15944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:16.173552 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 15734
I20260811 00:24:16.198072 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 15088
I20260811 00:24:16.199337 15200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:16.314893 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 15088
2026-08-11T00:24:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4995 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T00:24:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:16Z Disabled control of system clock
I20260811 00:24:16.422670 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:46033
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:24:16.810343 16055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:16.810696 16055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:16.810786 16055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:16.819389 16055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:16.819563 16055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:16.819617 16055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:16.819700 16055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:16.819746 16055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:16.829682 16055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:46033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:46033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:16.831075 16055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:16.833434 16055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:16.846542 16069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:16.846637 16067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:16.846673 16066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:16.847901 16055 server_base.cc:1034] running on GCE node
I20260811 00:24:16.848914 16055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:16.850548 16055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:16.851807 16055 hybrid_clock.cc:648] HybridClock initialized: now 1786407856851746 us; error 58 us; skew 500 ppm
I20260811 00:24:16.852269 16055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:16.854990 16055 webserver.cc:460] Webserver started at http://127.16.71.190:42209/ using document root <none> and password file <none>
I20260811 00:24:16.855957 16055 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:16.856093 16055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:16.856451 16055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:16.859901 16055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "77e84d29dbd947648b9fa13c7af750fa"
format_stamp: "Formatted at 2026-08-11 00:24:16 on dist-test-slave-f7th"
server_key: "d4c2e2f0f444321bd9c382978a2d9313"
server_key_iv: "741b23ca00f1e87320b4cc5446df33b5"
server_key_version: "encryptionkey@0"
I20260811 00:24:16.860895 16055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "77e84d29dbd947648b9fa13c7af750fa"
format_stamp: "Formatted at 2026-08-11 00:24:16 on dist-test-slave-f7th"
server_key: "d4c2e2f0f444321bd9c382978a2d9313"
server_key_iv: "741b23ca00f1e87320b4cc5446df33b5"
server_key_version: "encryptionkey@0"
I20260811 00:24:16.867537 16055 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260811 00:24:16.871881 16083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:16.873560 16055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:24:16.873769 16055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "77e84d29dbd947648b9fa13c7af750fa"
format_stamp: "Formatted at 2026-08-11 00:24:16 on dist-test-slave-f7th"
server_key: "d4c2e2f0f444321bd9c382978a2d9313"
server_key_iv: "741b23ca00f1e87320b4cc5446df33b5"
server_key_version: "encryptionkey@0"
I20260811 00:24:16.874027 16055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:16.899688 16055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:16.900733 16055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:16.901058 16055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:16.921792 16055 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:46033
I20260811 00:24:16.921811 16181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:46033 every 8 connection(s)
I20260811 00:24:16.923308 16055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:16.927603 16182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:16.929458 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 16055
I20260811 00:24:16.929679 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:16.930186 16670 external_mini_cluster.cc:1428] Setting key fee8c8dade6e1831f3e9a8bda007b939
I20260811 00:24:16.940752 16182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa: Bootstrap starting.
I20260811 00:24:16.944478 16182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:16.945889 16182 log.cc:824] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:16.946159 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:16.932441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.16.71.190:46033)
0811 00:24:16.933799 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:16.933813 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:16.933872 (+ 59us) server_negotiation.cc:407] Connection header received
0811 00:24:16.935777 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:16.935806 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:16.936478 (+ 672us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:16.937057 (+ 579us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:16.937747 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:16.939218 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:16.940751 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:16.941568 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:16.941799 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:16.943172 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:16.943181 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:16.943198 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:16.943845 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:16.943963 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:16.943988 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:16.944105 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:16.944367 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 00:24:16.944817 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":506,"threads_started":1}
I20260811 00:24:16.949975 16182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa: No bootstrap required, opened a new log
I20260811 00:24:16.958774 16182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e84d29dbd947648b9fa13c7af750fa" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 46033 } }
I20260811 00:24:16.959247 16182 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:16.959327 16182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e84d29dbd947648b9fa13c7af750fa, State: Initialized, Role: FOLLOWER
I20260811 00:24:16.960070 16182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e84d29dbd947648b9fa13c7af750fa" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 46033 } }
I20260811 00:24:16.960407 16182 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:16.960588 16182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:16.960827 16182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:16.963306 16182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e84d29dbd947648b9fa13c7af750fa" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 46033 } }
I20260811 00:24:16.963915 16182 leader_election.cc:302] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77e84d29dbd947648b9fa13c7af750fa; no voters:
I20260811 00:24:16.964574 16182 leader_election.cc:288] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:16.964768 16191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:16.965623 16191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [term 1 LEADER]: Becoming Leader. State: Replica: 77e84d29dbd947648b9fa13c7af750fa, State: Running, Role: LEADER
I20260811 00:24:16.966245 16191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e84d29dbd947648b9fa13c7af750fa" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 46033 } }
I20260811 00:24:16.967473 16182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:16.969604 16194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77e84d29dbd947648b9fa13c7af750fa. Latest consensus state: current_term: 1 leader_uuid: "77e84d29dbd947648b9fa13c7af750fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e84d29dbd947648b9fa13c7af750fa" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 46033 } } }
I20260811 00:24:16.970084 16194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:16.970727 16192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77e84d29dbd947648b9fa13c7af750fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e84d29dbd947648b9fa13c7af750fa" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 46033 } } }
I20260811 00:24:16.971251 16192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:16.972868 16203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:16.979460 16203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:16.990792 16203 catalog_manager.cc:1347] Generated new cluster ID: 45ec50ee00304bbba57328f335015369
I20260811 00:24:16.990928 16203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:17.008363 16203 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:17.010210 16203 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:17.018646 16203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa: Generated new TSK 0
I20260811 00:24:17.019721 16203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:17.035400 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:46033
--builtin_ntp_servers=127.16.71.148:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:24:17.460099 16227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:17.460474 16227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:17.460659 16227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:17.468516 16227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:17.468664 16227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:17.468837 16227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:24:17.479123 16227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:46033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:17.480826 16227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:17.482815 16227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:17.496726 16236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:17.496981 16242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:17.496713 16237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:17.497350 16227 server_base.cc:1034] running on GCE node
I20260811 00:24:17.498143 16227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:17.499203 16227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:17.500448 16227 hybrid_clock.cc:648] HybridClock initialized: now 1786407857500389 us; error 46 us; skew 500 ppm
I20260811 00:24:17.500825 16227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:17.503237 16227 webserver.cc:460] Webserver started at http://127.16.71.129:43585/ using document root <none> and password file <none>
I20260811 00:24:17.504051 16227 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:17.504168 16227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:17.504492 16227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:17.507640 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "274a3602d2724632851ce8984b5fe76d"
format_stamp: "Formatted at 2026-08-11 00:24:17 on dist-test-slave-f7th"
server_key: "1f8e360d508a3d4c455b47e778b1abf8"
server_key_iv: "2fce65702b64056c6197114ac41c169c"
server_key_version: "encryptionkey@0"
I20260811 00:24:17.508490 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "274a3602d2724632851ce8984b5fe76d"
format_stamp: "Formatted at 2026-08-11 00:24:17 on dist-test-slave-f7th"
server_key: "1f8e360d508a3d4c455b47e778b1abf8"
server_key_iv: "2fce65702b64056c6197114ac41c169c"
server_key_version: "encryptionkey@0"
I20260811 00:24:17.514214 16227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:24:17.518277 16253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:17.520026 16227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:24:17.520241 16227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "274a3602d2724632851ce8984b5fe76d"
format_stamp: "Formatted at 2026-08-11 00:24:17 on dist-test-slave-f7th"
server_key: "1f8e360d508a3d4c455b47e778b1abf8"
server_key_iv: "2fce65702b64056c6197114ac41c169c"
server_key_version: "encryptionkey@0"
I20260811 00:24:17.520488 16227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:17.543965 16227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:17.544941 16227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:17.545253 16227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:17.546394 16227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:17.548072 16227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:17.548171 16227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:17.548282 16227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:17.548355 16227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:17.580457 16227 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:38679
I20260811 00:24:17.580536 16442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:38679 every 8 connection(s)
I20260811 00:24:17.581883 16227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:24:17.591066 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 16227
I20260811 00:24:17.591272 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:24:17.591724 16670 external_mini_cluster.cc:1428] Setting key 35a41c277aa017666f716dcd529b81d2
I20260811 00:24:17.594820 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:46033
--builtin_ntp_servers=127.16.71.148:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:24:17.597877 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:17.586966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:44971 (local address 127.16.71.190:46033)
0811 00:24:17.587451 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:17.587458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:17.588817 (+ 1359us) server_negotiation.cc:407] Connection header received
0811 00:24:17.590714 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:17.590720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:17.590881 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:17.591017 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:17.592207 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:17.593147 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:17.594945 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:17.595406 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:17.595522 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:17.597031 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:17.597036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:17.597039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:17.597058 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:17.597099 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:17.597111 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:17.597189 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:17.597420 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 00:24:17.597529 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260811 00:24:17.599022 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:17.587649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:46033 (local address 127.16.71.129:44971)
0811 00:24:17.588654 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:17.588701 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:17.590377 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:17.591208 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:17.591220 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:17.591326 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:17.592021 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:17.592042 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:17.593356 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:17.593360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:17.594792 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:17.594807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:17.595651 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:17.595660 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:17.595898 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:17.596646 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:17.596650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:17.596703 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:17.596802 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:17.597209 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:17.597221 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:17.597228 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:24:17.597395 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 00:24:17.597853 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":275,"threads_started":1}
I20260811 00:24:17.600694 16443 heartbeater.cc:347] Connected to a master server at 127.16.71.190:46033
I20260811 00:24:17.600998 16443 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:17.601783 16443 heartbeater.cc:511] Master 127.16.71.190:46033 requested a full tablet report, sending...
I20260811 00:24:17.604110 16110 ts_manager.cc:194] Registered new tserver with Master: 274a3602d2724632851ce8984b5fe76d (127.16.71.129:38679)
I20260811 00:24:17.606052 16110 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.129:44971
W20260811 00:24:17.968164 16450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:17.968495 16450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:17.968652 16450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:17.976321 16450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:17.976449 16450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:17.976567 16450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:24:17.986335 16450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:46033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:17.987730 16450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:17.989760 16450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:18.003684 16458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:18.003625 16461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:18.003695 16459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:18.004689 16450 server_base.cc:1034] running on GCE node
I20260811 00:24:18.005426 16450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:18.006559 16450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:18.007792 16450 hybrid_clock.cc:648] HybridClock initialized: now 1786407858007726 us; error 60 us; skew 500 ppm
I20260811 00:24:18.008132 16450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:18.010692 16450 webserver.cc:460] Webserver started at http://127.16.71.130:34693/ using document root <none> and password file <none>
I20260811 00:24:18.011402 16450 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:18.011528 16450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:18.011845 16450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:18.014962 16450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "fb00c592683d4bafa27ca3807c27e2a6"
format_stamp: "Formatted at 2026-08-11 00:24:18 on dist-test-slave-f7th"
server_key: "55e502ce158bb23c1a96d9b14f28d353"
server_key_iv: "1615dd53385ecda30461ea7e3c716404"
server_key_version: "encryptionkey@0"
I20260811 00:24:18.015839 16450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "fb00c592683d4bafa27ca3807c27e2a6"
format_stamp: "Formatted at 2026-08-11 00:24:18 on dist-test-slave-f7th"
server_key: "55e502ce158bb23c1a96d9b14f28d353"
server_key_iv: "1615dd53385ecda30461ea7e3c716404"
server_key_version: "encryptionkey@0"
I20260811 00:24:18.022047 16450 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 00:24:18.026162 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:18.027823 16450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:24:18.028036 16450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "fb00c592683d4bafa27ca3807c27e2a6"
format_stamp: "Formatted at 2026-08-11 00:24:18 on dist-test-slave-f7th"
server_key: "55e502ce158bb23c1a96d9b14f28d353"
server_key_iv: "1615dd53385ecda30461ea7e3c716404"
server_key_version: "encryptionkey@0"
I20260811 00:24:18.028296 16450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:18.076064 16450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:18.077163 16450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:18.077508 16450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:18.078646 16450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:18.080457 16450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:18.080564 16450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:18.080698 16450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:18.080775 16450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:18.114001 16450 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:45713
I20260811 00:24:18.114060 16669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:45713 every 8 connection(s)
I20260811 00:24:18.115365 16450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:24:18.121347 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 16450
I20260811 00:24:18.121577 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:24:18.122167 16670 external_mini_cluster.cc:1428] Setting key 7fcf28e43fa1981630bcf39b6502f979
I20260811 00:24:18.125861 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:46033
--builtin_ntp_servers=127.16.71.148:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:24:18.131031 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.118278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:42271 (local address 127.16.71.190:46033)
0811 00:24:18.118800 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.118806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.120235 (+ 1429us) server_negotiation.cc:407] Connection header received
0811 00:24:18.122805 (+ 2570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.122812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.122941 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.123087 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:18.124825 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.125910 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.127802 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.128367 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.128497 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.130163 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:18.130169 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:18.130173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:18.130199 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:18.130250 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:18.130265 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:18.130350 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.130576 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.130686 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
I20260811 00:24:18.132164 16684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.119014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:46033 (local address 127.16.71.130:42271)
0811 00:24:18.120051 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:18.120091 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:18.122382 (+ 2291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:18.123566 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:18.123576 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:18.123692 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:18.124531 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.124580 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.126143 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.126150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.127596 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.127614 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.128668 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.128677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.128974 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.129770 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:18.129773 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:18.129823 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:18.129919 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:18.130363 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:18.130370 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:18.130377 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:24:18.130527 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 00:24:18.130886 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":314,"threads_started":1}
I20260811 00:24:18.133901 16676 heartbeater.cc:347] Connected to a master server at 127.16.71.190:46033
I20260811 00:24:18.134274 16676 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:18.135265 16676 heartbeater.cc:511] Master 127.16.71.190:46033 requested a full tablet report, sending...
I20260811 00:24:18.137102 16110 ts_manager.cc:194] Registered new tserver with Master: fb00c592683d4bafa27ca3807c27e2a6 (127.16.71.130:45713)
I20260811 00:24:18.138090 16110 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.130:42271
W20260811 00:24:18.487505 16685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:18.487844 16685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:18.487982 16685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:18.495733 16685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:18.495879 16685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:18.496016 16685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:24:18.505823 16685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:46033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:18.507328 16685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:18.509331 16685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:18.524369 16698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:18.524480 16691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:18.524679 16696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:18.525693 16685 server_base.cc:1034] running on GCE node
I20260811 00:24:18.526378 16685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:18.527504 16685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:18.528770 16685 hybrid_clock.cc:648] HybridClock initialized: now 1786407858528701 us; error 57 us; skew 500 ppm
I20260811 00:24:18.529143 16685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:18.532339 16685 webserver.cc:460] Webserver started at http://127.16.71.131:41755/ using document root <none> and password file <none>
I20260811 00:24:18.533218 16685 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:18.533367 16685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:18.533731 16685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:18.537642 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "c25d22543384492c816543872e48ddd8"
format_stamp: "Formatted at 2026-08-11 00:24:18 on dist-test-slave-f7th"
server_key: "39702eb2e684f850242169a268e353f9"
server_key_iv: "d1c71407656fc87cf3b66cf9385b7e8f"
server_key_version: "encryptionkey@0"
I20260811 00:24:18.539198 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "c25d22543384492c816543872e48ddd8"
format_stamp: "Formatted at 2026-08-11 00:24:18 on dist-test-slave-f7th"
server_key: "39702eb2e684f850242169a268e353f9"
server_key_iv: "d1c71407656fc87cf3b66cf9385b7e8f"
server_key_version: "encryptionkey@0"
I20260811 00:24:18.546551 16685 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.007s
I20260811 00:24:18.550817 16708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:18.552531 16685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:24:18.552790 16685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "c25d22543384492c816543872e48ddd8"
format_stamp: "Formatted at 2026-08-11 00:24:18 on dist-test-slave-f7th"
server_key: "39702eb2e684f850242169a268e353f9"
server_key_iv: "d1c71407656fc87cf3b66cf9385b7e8f"
server_key_version: "encryptionkey@0"
I20260811 00:24:18.553057 16685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:18.583573 16685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:18.584839 16685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:18.585219 16685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:18.586717 16685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:18.588742 16685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:18.588850 16685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:18.588948 16685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:18.589008 16685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:18.609800 16443 heartbeater.cc:503] Master 127.16.71.190:46033 was elected leader, sending a full tablet report...
I20260811 00:24:18.622216 16685 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:34721
I20260811 00:24:18.622309 16898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:34721 every 8 connection(s)
I20260811 00:24:18.623696 16685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:24:18.630779 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 16685
I20260811 00:24:18.630999 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:24:18.631577 16670 external_mini_cluster.cc:1428] Setting key 135a0498ccaed27a0e0b438842c979d3
I20260811 00:24:18.639743 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.627452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:57095 (local address 127.16.71.190:46033)
0811 00:24:18.627679 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.627685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.629484 (+ 1799us) server_negotiation.cc:407] Connection header received
0811 00:24:18.631938 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.631944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.632036 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.632169 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:18.633752 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.634763 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.636672 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.637106 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.637221 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.638869 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:18.638876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:18.638879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:18.638901 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:18.638939 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:18.638952 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:18.639034 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.639362 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.639477 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:24:18.640978 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.628148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:46033 (local address 127.16.71.131:57095)
0811 00:24:18.629293 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:18.629334 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:18.631535 (+ 2201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:18.632424 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:18.632438 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:18.632638 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:18.633530 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.633569 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.635041 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.635048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.636447 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.636464 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.637390 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.637398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.637668 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.638451 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:18.638454 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:18.638502 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:18.638602 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:18.639097 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:18.639107 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:18.639114 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:24:18.639302 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 00:24:18.639678 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":402,"threads_started":1}
I20260811 00:24:18.642601 16899 heartbeater.cc:347] Connected to a master server at 127.16.71.190:46033
I20260811 00:24:18.642938 16899 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:18.643785 16899 heartbeater.cc:511] Master 127.16.71.190:46033 requested a full tablet report, sending...
I20260811 00:24:18.645431 16110 ts_manager.cc:194] Registered new tserver with Master: c25d22543384492c816543872e48ddd8 (127.16.71.131:34721)
I20260811 00:24:18.646381 16110 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.131:57095
I20260811 00:24:18.648072 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:24:18.659904 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.652983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.16.71.190:46033)
0811 00:24:18.653228 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.653235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.653582 (+ 347us) server_negotiation.cc:407] Connection header received
0811 00:24:18.653868 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.653874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.653967 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.654094 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:18.654895 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.655681 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.657382 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.657829 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.657955 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.659113 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:18.659118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:18.659122 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:18.659145 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:18.659184 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:18.659197 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:18.659281 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.659430 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.659568 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:24:18.664372 16110 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:24:18.672103 16110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:24:18.712131 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.687836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.16.71.130:45713)
0811 00:24:18.700606 (+ 12770us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.700639 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.702256 (+ 1617us) server_negotiation.cc:407] Connection header received
0811 00:24:18.702485 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.702494 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.702900 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.703264 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:18.703911 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.705489 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.708638 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.710482 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.710669 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.710971 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.711581 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.711780 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12522,"thread_start_us":12790,"threads_started":1}
I20260811 00:24:18.712157 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.687340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:45713 (local address 127.0.0.1:47148)
0811 00:24:18.702108 (+ 14768us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:18.702132 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:18.702302 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:18.703204 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:18.703212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:18.703244 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:18.703706 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.703729 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.705642 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.705647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.708307 (+ 2660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.708328 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.711130 (+ 2802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.711137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.711308 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.711346 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:24:18.711499 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 00:24:18.711757 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14615,"mutex_wait_us":157,"thread_start_us":14163,"threads_started":1}
I20260811 00:24:18.712531 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.687339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:38679 (local address 127.0.0.1:40516)
0811 00:24:18.696847 (+ 9508us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:18.696932 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:18.697214 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:18.703607 (+ 6393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:18.703611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:18.703639 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:18.704005 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.704023 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.705838 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.705844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.709030 (+ 3186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.709051 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.711585 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.711591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.711839 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.711888 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 00:24:18.711985 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 00:24:18.712169 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9045,"mutex_wait_us":148,"thread_start_us":15271,"threads_started":1}
I20260811 00:24:18.712970 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.689027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46778 (local address 127.16.71.131:34721)
0811 00:24:18.700935 (+ 11908us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.700944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.703018 (+ 2074us) server_negotiation.cc:407] Connection header received
0811 00:24:18.703183 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.703190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.703417 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.703639 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:18.704532 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.706510 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.709545 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.710938 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.711102 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.711383 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.712127 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.712596 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11734,"thread_start_us":11598,"threads_started":1}
I20260811 00:24:18.713665 16923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.689011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:34721 (local address 127.0.0.1:46778)
0811 00:24:18.702859 (+ 13848us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:18.702876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:18.703015 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:18.703924 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:18.703932 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:18.703962 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:18.704325 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.704338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.706713 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.706717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.709260 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.709276 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.711569 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.711577 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.711833 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.711892 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 00:24:18.711985 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:24:18.713306 (+ 1321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13734,"mutex_wait_us":1231,"thread_start_us":7179,"threads_started":1}
I20260811 00:24:18.713820 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.687392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.16.71.129:38679)
0811 00:24:18.701648 (+ 14256us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.701658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.701700 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:24:18.701868 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.701879 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.703024 (+ 1145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.703344 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:18.704256 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.705673 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.709367 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.710945 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.711097 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.711383 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.713334 (+ 1951us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.713503 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14049,"thread_start_us":13909,"threads_started":1}
I20260811 00:24:18.718294 16338 tablet_service.cc:1467] Processing CreateTablet for tablet 712c339df5c3440e98136b78c67b6d4b (DEFAULT_TABLE table=test-table [id=e8344cd45cd54c74b3e8ce81de1ce46c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:18.718102 16549 tablet_service.cc:1467] Processing CreateTablet for tablet 712c339df5c3440e98136b78c67b6d4b (DEFAULT_TABLE table=test-table [id=e8344cd45cd54c74b3e8ce81de1ce46c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:18.718523 16796 tablet_service.cc:1467] Processing CreateTablet for tablet 712c339df5c3440e98136b78c67b6d4b (DEFAULT_TABLE table=test-table [id=e8344cd45cd54c74b3e8ce81de1ce46c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:18.719944 16549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 712c339df5c3440e98136b78c67b6d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:18.719944 16338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 712c339df5c3440e98136b78c67b6d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:18.720036 16796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 712c339df5c3440e98136b78c67b6d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:18.734285 16943 tablet_bootstrap.cc:492] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Bootstrap starting.
I20260811 00:24:18.736650 16944 tablet_bootstrap.cc:492] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Bootstrap starting.
I20260811 00:24:18.738425 16945 tablet_bootstrap.cc:492] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Bootstrap starting.
I20260811 00:24:18.740782 16943 tablet_bootstrap.cc:654] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:18.740785 16944 tablet_bootstrap.cc:654] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:18.741454 16945 tablet_bootstrap.cc:654] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:18.742187 16943 log.cc:824] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:18.742187 16944 log.cc:824] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:18.742475 16945 log.cc:824] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:18.745761 16943 tablet_bootstrap.cc:492] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: No bootstrap required, opened a new log
I20260811 00:24:18.745972 16944 tablet_bootstrap.cc:492] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: No bootstrap required, opened a new log
I20260811 00:24:18.745966 16945 tablet_bootstrap.cc:492] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: No bootstrap required, opened a new log
I20260811 00:24:18.746047 16943 ts_tablet_manager.cc:1397] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260811 00:24:18.746254 16945 ts_tablet_manager.cc:1397] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 00:24:18.746250 16944 ts_tablet_manager.cc:1397] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260811 00:24:18.753492 16943 raft_consensus.cc:348] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.753913 16943 raft_consensus.cc:374] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:18.754009 16943 raft_consensus.cc:729] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25d22543384492c816543872e48ddd8, State: Initialized, Role: FOLLOWER
I20260811 00:24:18.754658 16943 consensus_queue.cc:260] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.756983 16944 raft_consensus.cc:348] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.756968 16945 raft_consensus.cc:348] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.757486 16945 raft_consensus.cc:374] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:18.757486 16944 raft_consensus.cc:374] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:18.757592 16899 heartbeater.cc:503] Master 127.16.71.190:46033 was elected leader, sending a full tablet report...
I20260811 00:24:18.757598 16944 raft_consensus.cc:729] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274a3602d2724632851ce8984b5fe76d, State: Initialized, Role: FOLLOWER
I20260811 00:24:18.757598 16945 raft_consensus.cc:729] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb00c592683d4bafa27ca3807c27e2a6, State: Initialized, Role: FOLLOWER
I20260811 00:24:18.758240 16944 consensus_queue.cc:260] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.758424 16945 consensus_queue.cc:260] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.760277 16944 ts_tablet_manager.cc:1428] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 00:24:18.761739 16676 heartbeater.cc:503] Master 127.16.71.190:46033 was elected leader, sending a full tablet report...
I20260811 00:24:18.762529 16945 ts_tablet_manager.cc:1428] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260811 00:24:18.762706 16943 ts_tablet_manager.cc:1428] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Time spent starting tablet: real 0.017s user 0.009s sys 0.003s
I20260811 00:24:18.796003 16952 raft_consensus.cc:482] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:24:18.796414 16952 raft_consensus.cc:504] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.798349 16952 leader_election.cc:288] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 274a3602d2724632851ce8984b5fe76d (127.16.71.129:38679), c25d22543384492c816543872e48ddd8 (127.16.71.131:34721)
I20260811 00:24:18.807700 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.798962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:42815 (local address 127.16.71.129:38679)
0811 00:24:18.799205 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.799211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.800229 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 00:24:18.800446 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.800451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.800556 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.800673 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:18.801266 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.802295 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.805546 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.806548 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.806723 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.806806 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.807248 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.807392 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 00:24:18.807698 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.798996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:44995 (local address 127.16.71.131:34721)
0811 00:24:18.799195 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.799201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.799823 (+ 622us) server_negotiation.cc:407] Connection header received
0811 00:24:18.800079 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.800083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.800147 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.800361 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:18.801002 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.802169 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.805376 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.806546 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.806730 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.806824 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.807283 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.807424 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 00:24:18.807982 16954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.798844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:38679 (local address 127.16.71.130:42815)
0811 00:24:18.800099 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:18.800115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:18.800258 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:18.800696 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:18.800701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:18.800732 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:18.801112 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.801123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.802446 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.802450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.805094 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.805118 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.806864 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.806874 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.807056 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.807101 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:24:18.807192 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:24:18.807391 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"mutex_wait_us":178,"thread_start_us":836,"threads_started":1}
I20260811 00:24:18.808022 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.798713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:34721 (local address 127.16.71.130:44995)
0811 00:24:18.799689 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:18.799708 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:18.799902 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:18.800400 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:18.800404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:18.800426 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:18.800814 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.800831 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.802349 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.802355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.805095 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:18.805118 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.806796 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.806804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:18.807044 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.807097 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 00:24:18.807190 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:24:18.807391 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":245,"thread_start_us":645,"threads_started":1}
I20260811 00:24:18.809264 16369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712c339df5c3440e98136b78c67b6d4b" candidate_uuid: "fb00c592683d4bafa27ca3807c27e2a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274a3602d2724632851ce8984b5fe76d" is_pre_election: true
I20260811 00:24:18.809473 16824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712c339df5c3440e98136b78c67b6d4b" candidate_uuid: "fb00c592683d4bafa27ca3807c27e2a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25d22543384492c816543872e48ddd8" is_pre_election: true
I20260811 00:24:18.810420 16824 raft_consensus.cc:2454] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb00c592683d4bafa27ca3807c27e2a6 in term 0.
I20260811 00:24:18.810214 16369 raft_consensus.cc:2454] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb00c592683d4bafa27ca3807c27e2a6 in term 0.
I20260811 00:24:18.811436 16473 leader_election.cc:302] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c25d22543384492c816543872e48ddd8, fb00c592683d4bafa27ca3807c27e2a6; no voters:
I20260811 00:24:18.811978 16952 raft_consensus.cc:2781] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:24:18.812147 16952 raft_consensus.cc:482] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:24:18.812268 16952 raft_consensus.cc:3037] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:18.814806 16952 raft_consensus.cc:504] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.815572 16952 leader_election.cc:288] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [CANDIDATE]: Term 1 election: Requested vote from peers 274a3602d2724632851ce8984b5fe76d (127.16.71.129:38679), c25d22543384492c816543872e48ddd8 (127.16.71.131:34721)
I20260811 00:24:18.816284 16369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712c339df5c3440e98136b78c67b6d4b" candidate_uuid: "fb00c592683d4bafa27ca3807c27e2a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274a3602d2724632851ce8984b5fe76d"
I20260811 00:24:18.816293 16824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712c339df5c3440e98136b78c67b6d4b" candidate_uuid: "fb00c592683d4bafa27ca3807c27e2a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25d22543384492c816543872e48ddd8"
I20260811 00:24:18.816520 16824 raft_consensus.cc:3037] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:18.816521 16369 raft_consensus.cc:3037] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:18.819227 16369 raft_consensus.cc:2454] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb00c592683d4bafa27ca3807c27e2a6 in term 1.
I20260811 00:24:18.819510 16824 raft_consensus.cc:2454] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb00c592683d4bafa27ca3807c27e2a6 in term 1.
I20260811 00:24:18.819871 16475 leader_election.cc:302] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274a3602d2724632851ce8984b5fe76d, fb00c592683d4bafa27ca3807c27e2a6; no voters:
I20260811 00:24:18.820322 16952 raft_consensus.cc:2781] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:18.820861 16952 raft_consensus.cc:686] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 1 LEADER]: Becoming Leader. State: Replica: fb00c592683d4bafa27ca3807c27e2a6, State: Running, Role: LEADER
I20260811 00:24:18.821442 16952 consensus_queue.cc:237] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } }
I20260811 00:24:18.828130 16106 catalog_manager.cc:5495] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb00c592683d4bafa27ca3807c27e2a6 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "fb00c592683d4bafa27ca3807c27e2a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb00c592683d4bafa27ca3807c27e2a6" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 } health_report { overall_health: UNKNOWN } } }
W20260811 00:24:18.836998 16446 tablet.cc:2369] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:24:18.867689 16677 tablet.cc:2369] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:18.874545 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.868516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47160 (local address 127.16.71.130:45713)
0811 00:24:18.868789 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.868793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.868814 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:24:18.868971 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.868974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.869081 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.869214 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:24:18.869608 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.870352 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.871896 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.872367 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.872487 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.873251 (+ 764us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:24:18.873814 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:24:18.874126 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.874174 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.874257 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260811 00:24:18.876353 16904 tablet.cc:2369] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:18.883425 16824 raft_consensus.cc:1261] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 1 FOLLOWER]: Refusing update from remote peer fb00c592683d4bafa27ca3807c27e2a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:18.883469 16369 raft_consensus.cc:1261] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 1 FOLLOWER]: Refusing update from remote peer fb00c592683d4bafa27ca3807c27e2a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:18.884488 16955 consensus_queue.cc:1035] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c25d22543384492c816543872e48ddd8" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 34721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:18.884931 16952 consensus_queue.cc:1035] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "274a3602d2724632851ce8984b5fe76d" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 38679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:18.913539 16106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:18.914258 16106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:18.918956 16106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77e84d29dbd947648b9fa13c7af750fa: Sending DeleteTablet for 3 replicas of tablet 712c339df5c3440e98136b78c67b6d4b
I20260811 00:24:18.920353 16796 tablet_service.cc:1514] Processing DeleteTablet for tablet 712c339df5c3440e98136b78c67b6d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:18 UTC) from {username='slave'} at 127.0.0.1:46778
I20260811 00:24:18.920414 16549 tablet_service.cc:1514] Processing DeleteTablet for tablet 712c339df5c3440e98136b78c67b6d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:18 UTC) from {username='slave'} at 127.0.0.1:47148
I20260811 00:24:18.921458 16338 tablet_service.cc:1514] Processing DeleteTablet for tablet 712c339df5c3440e98136b78c67b6d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:18 UTC) from {username='slave'} at 127.0.0.1:40516
I20260811 00:24:18.929976 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 16227
I20260811 00:24:18.930763 16979 tablet_replica.cc:330] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: stopping tablet replica
I20260811 00:24:18.931434 16979 raft_consensus.cc:2229] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:18.932199 16979 raft_consensus.cc:2258] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:18.938022 16973 tablet_replica.cc:330] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: stopping tablet replica
I20260811 00:24:18.938571 16973 raft_consensus.cc:2229] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:18.938647 16979 ts_tablet_manager.cc:1905] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:18.939361 16973 raft_consensus.cc:2258] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:18.942513 16973 ts_tablet_manager.cc:1905] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:18.943356 16978 tablet_replica.cc:330] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: stopping tablet replica
I20260811 00:24:18.943949 16978 raft_consensus.cc:2229] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:24:18.944805 16978 raft_consensus.cc:2258] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:18.947508 16978 ts_tablet_manager.cc:1905] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:18.951005 16979 ts_tablet_manager.cc:1918] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:18.951005 16973 ts_tablet_manager.cc:1918] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:18.951241 16973 log.cc:1196] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/712c339df5c3440e98136b78c67b6d4b
I20260811 00:24:18.951241 16979 log.cc:1196] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/712c339df5c3440e98136b78c67b6d4b
I20260811 00:24:18.951869 16979 ts_tablet_manager.cc:1939] T 712c339df5c3440e98136b78c67b6d4b P 274a3602d2724632851ce8984b5fe76d: Deleting consensus metadata
I20260811 00:24:18.951858 16973 ts_tablet_manager.cc:1939] T 712c339df5c3440e98136b78c67b6d4b P c25d22543384492c816543872e48ddd8: Deleting consensus metadata
I20260811 00:24:18.954133 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:18.931729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40518 (local address 127.16.71.129:38679)
0811 00:24:18.931933 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:18.931939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:18.934934 (+ 2995us) server_negotiation.cc:407] Connection header received
0811 00:24:18.935172 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:18.935179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:18.935332 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:18.935448 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:18.945512 (+ 10064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.946931 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.949266 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:18.950085 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:18.950215 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:18.952107 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:18.952125 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:18.952148 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:18.953095 (+ 947us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:18.953166 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:18.953199 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:18.953323 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:18.953614 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 00:24:18.953758 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:24:18.954104 16087 catalog_manager.cc:4841] TS 274a3602d2724632851ce8984b5fe76d (127.16.71.129:38679): tablet 712c339df5c3440e98136b78c67b6d4b (table test-table [id=e8344cd45cd54c74b3e8ce81de1ce46c]) successfully deleted
I20260811 00:24:18.954237 16084 catalog_manager.cc:4841] TS c25d22543384492c816543872e48ddd8 (127.16.71.131:34721): tablet 712c339df5c3440e98136b78c67b6d4b (table test-table [id=e8344cd45cd54c74b3e8ce81de1ce46c]) successfully deleted
I20260811 00:24:18.955529 16437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:18.961897 16978 ts_tablet_manager.cc:1918] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:18.962291 16978 log.cc:1196] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/712c339df5c3440e98136b78c67b6d4b
I20260811 00:24:18.963088 16978 ts_tablet_manager.cc:1939] T 712c339df5c3440e98136b78c67b6d4b P fb00c592683d4bafa27ca3807c27e2a6: Deleting consensus metadata
I20260811 00:24:18.965608 16089 catalog_manager.cc:4841] TS fb00c592683d4bafa27ca3807c27e2a6 (127.16.71.130:45713): tablet 712c339df5c3440e98136b78c67b6d4b (table test-table [id=e8344cd45cd54c74b3e8ce81de1ce46c]) successfully deleted
I20260811 00:24:19.051548 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 16227
I20260811 00:24:19.079564 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 16450
I20260811 00:24:19.086941 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:19.080361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.16.71.130:45713)
0811 00:24:19.080601 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:19.080606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:19.080662 (+ 56us) server_negotiation.cc:407] Connection header received
0811 00:24:19.080840 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:19.080846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:19.080936 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:19.081049 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:19.081471 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:19.082302 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:19.083701 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:19.084166 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:19.084285 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:19.085407 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:19.085417 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:19.085433 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:19.086089 (+ 656us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:19.086141 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:19.086167 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:19.086266 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:19.086421 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 00:24:19.086560 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 00:24:19.087720 16662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:19.202212 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 16450
I20260811 00:24:19.234802 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 16685
I20260811 00:24:19.242529 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:19.235756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.16.71.131:34721)
0811 00:24:19.236001 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:19.236008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:19.236031 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:24:19.236134 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:19.236140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:19.236235 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:19.236333 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:19.236744 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:19.237542 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:19.238917 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:19.239351 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:19.239472 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:19.240802 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:19.240814 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:19.240832 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:19.241664 (+ 832us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:19.241723 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:19.241749 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:19.241894 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:19.242072 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 00:24:19.242206 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 00:24:19.243234 16893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:19.340183 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 16685
I20260811 00:24:19.365295 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 16055
I20260811 00:24:19.366395 16171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:19.481906 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 16055
2026-08-11T00:24:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3171 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:19 dist-test-slave-f7th krb5kdc[16992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:19 dist-test-slave-f7th krb5kdc[16992](info): set up 2 sockets
Aug 11 00:24:19 dist-test-slave-f7th krb5kdc[16992](info): commencing operation
krb5kdc: starting...
W20260811 00:24:20.828289 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:21 dist-test-slave-f7th krb5kdc[16992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:24:21.173179 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39963
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:39963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:21.549414 17013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:21.549798 17013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:21.549940 17013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:21.559006 17013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:21.559156 17013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:21.559240 17013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:21.559331 17013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:21.559397 17013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:21.570379 17013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:39963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:21.571867 17013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:21.573364 17013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:21.586578 17019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:21.586522 17024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:21.586606 17020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:21.587628 17013 server_base.cc:1034] running on GCE node
I20260811 00:24:21.588536 17013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:21.590220 17013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:21.591486 17013 hybrid_clock.cc:648] HybridClock initialized: now 1786407861591419 us; error 51 us; skew 500 ppm
I20260811 00:24:21.591868 17013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:21 dist-test-slave-f7th krb5kdc[16992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407861, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:21.599473 17013 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:24:21.601409 17013 webserver.cc:460] Webserver started at http://127.16.71.190:40827/ using document root <none> and password file <none>
I20260811 00:24:21.602218 17013 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:21.602358 17013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:21.602689 17013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:21.606056 17013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb"
format_stamp: "Formatted at 2026-08-11 00:24:21 on dist-test-slave-f7th"
server_key: "01127642365988fd3612b990ab06039e"
server_key_iv: "69a49a9ff6080efa643b0925c76a7ec1"
server_key_version: "encryptionkey@0"
I20260811 00:24:21.606920 17013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb"
format_stamp: "Formatted at 2026-08-11 00:24:21 on dist-test-slave-f7th"
server_key: "01127642365988fd3612b990ab06039e"
server_key_iv: "69a49a9ff6080efa643b0925c76a7ec1"
server_key_version: "encryptionkey@0"
I20260811 00:24:21.612900 17013 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:24:21.616816 17034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:21.618599 17013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:24:21.618829 17013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb"
format_stamp: "Formatted at 2026-08-11 00:24:21 on dist-test-slave-f7th"
server_key: "01127642365988fd3612b990ab06039e"
server_key_iv: "69a49a9ff6080efa643b0925c76a7ec1"
server_key_version: "encryptionkey@0"
I20260811 00:24:21.619064 17013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:21.653750 17013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:21.658870 17013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:21.659241 17013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:21.679476 17013 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:39963
I20260811 00:24:21.679500 17124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:39963 every 8 connection(s)
I20260811 00:24:21.681144 17013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:21.685877 17125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:21.688974 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17013
I20260811 00:24:21.689204 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:21.689780 16670 external_mini_cluster.cc:1428] Setting key 2b385c681c73a2d71c3893ba812c29b4
I20260811 00:24:21.700798 17125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb: Bootstrap starting.
I20260811 00:24:21.705777 17125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:21.707350 17125 log.cc:824] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:24:21 dist-test-slave-f7th krb5kdc[16992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:21.711033 17125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb: No bootstrap required, opened a new log
I20260811 00:24:21.718727 17131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:21.692076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37940 (local address 127.16.71.190:39963)
0811 00:24:21.693050 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:21.693064 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:21.693114 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:24:21.694920 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:21.694948 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:21.695592 (+ 644us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:21.696118 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:21.698202 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:21.700429 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:21.702867 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:21.704071 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:21.704280 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:21.710580 (+ 6300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:21.710588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:21.710606 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:21.710650 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:21.713991 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:21.714840 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:21.714847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:21.714854 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:21.715059 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:21.715505 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:21.715509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:21.715511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:21.716138 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:21.716481 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:21.717006 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 00:24:21.717548 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":437,"threads_started":1}
I20260811 00:24:21.719849 17125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39963 } }
I20260811 00:24:21.720359 17125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:21.720477 17125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aafdfa17fbc45a085d0ecdc3c5da7bb, State: Initialized, Role: FOLLOWER
I20260811 00:24:21.721215 17125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39963 } }
I20260811 00:24:21.721530 17125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:21.721683 17125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:21.721885 17125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:21.724488 17125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39963 } }
I20260811 00:24:21.725170 17125 leader_election.cc:302] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0aafdfa17fbc45a085d0ecdc3c5da7bb; no voters:
I20260811 00:24:21.725805 17125 leader_election.cc:288] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:21.725975 17137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:21.726748 17137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [term 1 LEADER]: Becoming Leader. State: Replica: 0aafdfa17fbc45a085d0ecdc3c5da7bb, State: Running, Role: LEADER
I20260811 00:24:21.727342 17137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39963 } }
I20260811 00:24:21.728283 17125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:21.730490 17140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0aafdfa17fbc45a085d0ecdc3c5da7bb. Latest consensus state: current_term: 1 leader_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39963 } } }
I20260811 00:24:21.731010 17139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aafdfa17fbc45a085d0ecdc3c5da7bb" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 39963 } } }
I20260811 00:24:21.731225 17140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:21.731264 17139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:21.732903 17150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:21.738222 17150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:21.747567 17150 catalog_manager.cc:1347] Generated new cluster ID: 38ebe2e3dbd148f0bca551d79e72cd42
I20260811 00:24:21.747713 17150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:21.763331 17150 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:21.765471 17150 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:21.779102 17150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb: Generated new TSK 0
I20260811 00:24:21.780514 17150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:24:21.864347 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39963
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:22.247462 17169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:22.247795 17169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:22.247942 17169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:22.255653 17169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:22.255820 17169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:22.255992 17169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:24:22.266535 17169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:22.268065 17169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:22.269704 17169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:22.284058 17178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:22.284112 17179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:22.284035 17181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:22.284814 17169 server_base.cc:1034] running on GCE node
I20260811 00:24:22.285449 17169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:22.286514 17169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:22.287765 17169 hybrid_clock.cc:648] HybridClock initialized: now 1786407862287708 us; error 51 us; skew 500 ppm
I20260811 00:24:22.288156 17169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:22 dist-test-slave-f7th krb5kdc[16992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:22.291841 17169 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:24:22.293678 17169 webserver.cc:460] Webserver started at http://127.16.71.129:40581/ using document root <none> and password file <none>
I20260811 00:24:22.294467 17169 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:22.294593 17169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:22.294912 17169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:22.298213 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "627132988d3c4588ae064a6574cddf6f"
format_stamp: "Formatted at 2026-08-11 00:24:22 on dist-test-slave-f7th"
server_key: "d5d56be2d58305fe5ec05c56acb00e65"
server_key_iv: "939af117707262032b6c7442705340a9"
server_key_version: "encryptionkey@0"
I20260811 00:24:22.299053 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "627132988d3c4588ae064a6574cddf6f"
format_stamp: "Formatted at 2026-08-11 00:24:22 on dist-test-slave-f7th"
server_key: "d5d56be2d58305fe5ec05c56acb00e65"
server_key_iv: "939af117707262032b6c7442705340a9"
server_key_version: "encryptionkey@0"
I20260811 00:24:22.305658 17169 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:24:22.309970 17194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:22.311617 17169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260811 00:24:22.311805 17169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "627132988d3c4588ae064a6574cddf6f"
format_stamp: "Formatted at 2026-08-11 00:24:22 on dist-test-slave-f7th"
server_key: "d5d56be2d58305fe5ec05c56acb00e65"
server_key_iv: "939af117707262032b6c7442705340a9"
server_key_version: "encryptionkey@0"
I20260811 00:24:22.312021 17169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:22.328377 17169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:22.333818 17169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:22.334115 17169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:22.335224 17169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:22.336776 17169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:22.336856 17169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:22.336964 17169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:22.337026 17169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:22.369761 17169 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:46767
I20260811 00:24:22.369808 17386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:46767 every 8 connection(s)
I20260811 00:24:22.371230 17169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:24:22.371594 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17169
I20260811 00:24:22.371758 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:24:22.372325 16670 external_mini_cluster.cc:1428] Setting key ffff41c8ffa92fd474ea767c869a244f
Aug 11 00:24:22 dist-test-slave-f7th krb5kdc[16992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:22.394512 17393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:22.374146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:43505 (local address 127.16.71.190:39963)
0811 00:24:22.374849 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:22.374853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:22.375933 (+ 1080us) server_negotiation.cc:407] Connection header received
0811 00:24:22.377917 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:22.377922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:22.378013 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:22.378111 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:22.379842 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.380602 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.382473 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.382909 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.383033 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:22.387219 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:22.387225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:22.387228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:22.387265 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:22.390672 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:22.391687 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:22.391693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:22.391696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:22.391867 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:22.392402 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:22.392407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:22.392411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:22.392846 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:22.393000 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:22.393849 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 00:24:22.394161 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":483,"threads_started":1}
I20260811 00:24:22.395375 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:22.374934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39963 (local address 127.16.71.129:43505)
0811 00:24:22.375781 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:22.375820 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:22.377627 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:22.378384 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:22.378394 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:22.379004 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:22.379639 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:22.379659 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.380810 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.380814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:22.382315 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:22.382330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.383178 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.383186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:22.383432 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:22.384126 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:22.384150 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:22.386856 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:22.390954 (+ 4098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:22.390963 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:22.390979 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:22.391522 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:22.392022 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:22.392026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:22.392029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:22.392267 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:22.393025 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:22.393033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:22.393421 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 00:24:22.393677 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 00:24:22.394154 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":353,"threads_started":1}
I20260811 00:24:22.397112 17389 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39963
I20260811 00:24:22.397486 17389 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:22.398264 17389 heartbeater.cc:511] Master 127.16.71.190:39963 requested a full tablet report, sending...
I20260811 00:24:22.400715 17060 ts_manager.cc:194] Registered new tserver with Master: 627132988d3c4588ae064a6574cddf6f (127.16.71.129:46767)
I20260811 00:24:22.402266 17060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:43505
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:24:22.446187 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39963
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:22.812209 17398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:22.812533 17398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:22.812680 17398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:22.820812 17398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:22.820948 17398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:22.821113 17398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:24:22.831612 17398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:22.833120 17398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:22.834575 17398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:22.848873 17406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:22.849083 17412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:22.849129 17407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:22.849777 17398 server_base.cc:1034] running on GCE node
I20260811 00:24:22.850526 17398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:22.851724 17398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:22.852957 17398 hybrid_clock.cc:648] HybridClock initialized: now 1786407862852907 us; error 46 us; skew 500 ppm
I20260811 00:24:22.853334 17398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:22 dist-test-slave-f7th krb5kdc[16992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:22.856925 17398 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:24:22.858970 17398 webserver.cc:460] Webserver started at http://127.16.71.130:42871/ using document root <none> and password file <none>
I20260811 00:24:22.859768 17398 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:22.859917 17398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:22.860270 17398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:22.863653 17398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "458d95bd19b24224b799128b0f44c90c"
format_stamp: "Formatted at 2026-08-11 00:24:22 on dist-test-slave-f7th"
server_key: "047245d8c5b5fe01bdd4f2852e10a949"
server_key_iv: "62abefe64dcb8beac8471e2de36fa38c"
server_key_version: "encryptionkey@0"
I20260811 00:24:22.864569 17398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "458d95bd19b24224b799128b0f44c90c"
format_stamp: "Formatted at 2026-08-11 00:24:22 on dist-test-slave-f7th"
server_key: "047245d8c5b5fe01bdd4f2852e10a949"
server_key_iv: "62abefe64dcb8beac8471e2de36fa38c"
server_key_version: "encryptionkey@0"
I20260811 00:24:22.870795 17398 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:24:22.875201 17427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:22.876852 17398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:24:22.877074 17398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "458d95bd19b24224b799128b0f44c90c"
format_stamp: "Formatted at 2026-08-11 00:24:22 on dist-test-slave-f7th"
server_key: "047245d8c5b5fe01bdd4f2852e10a949"
server_key_iv: "62abefe64dcb8beac8471e2de36fa38c"
server_key_version: "encryptionkey@0"
I20260811 00:24:22.877300 17398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:22.916373 17398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:22.921042 17398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:22.921329 17398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:22.922437 17398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:22.923955 17398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:22.924057 17398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:22.924181 17398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:22.924233 17398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:22.958771 17398 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:34563
I20260811 00:24:22.958832 17627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:34563 every 8 connection(s)
I20260811 00:24:22.960166 17398 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:24:22.961218 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17398
I20260811 00:24:22.961413 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:24:22.961938 16670 external_mini_cluster.cc:1428] Setting key 2e586ff2ef9fd42b97fed8af043a8363
Aug 11 00:24:22 dist-test-slave-f7th krb5kdc[16992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:22.983371 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:22.963126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:43017 (local address 127.16.71.190:39963)
0811 00:24:22.963631 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:22.963637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:22.964795 (+ 1158us) server_negotiation.cc:407] Connection header received
0811 00:24:22.966976 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:22.966981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:22.967079 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:22.967181 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:22.969031 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.969824 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.971597 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.972060 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.972189 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:22.976105 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:22.976111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:22.976114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:22.976154 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:22.979431 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:22.980439 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:22.980444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:22.980446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:22.980644 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:22.981159 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:22.981162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:22.981165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:22.981573 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:22.981808 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:22.982652 (+ 844us) server_negotiation.cc:299] Negotiation successful
0811 00:24:22.982938 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":245,"threads_started":1}
I20260811 00:24:22.984609 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:22.963725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39963 (local address 127.16.71.130:43017)
0811 00:24:22.964589 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:22.964670 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:22.966604 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:22.967484 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:22.967494 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:22.968084 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:22.968801 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:22.968825 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.970023 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.970028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:22.971379 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:22.971397 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:22.972338 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:22.972346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:22.972608 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:22.973389 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:22.973416 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:22.975790 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:22.979637 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:22.979646 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:22.979664 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:22.980263 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:22.980771 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:22.980775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:22.980778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:22.981006 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:22.981736 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:22.981746 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:22.982191 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 00:24:22.982514 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 00:24:22.983184 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":379,"threads_started":1}
I20260811 00:24:22.986799 17630 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39963
I20260811 00:24:22.987226 17630 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:22.988104 17630 heartbeater.cc:511] Master 127.16.71.190:39963 requested a full tablet report, sending...
I20260811 00:24:22.990120 17060 ts_manager.cc:194] Registered new tserver with Master: 458d95bd19b24224b799128b0f44c90c (127.16.71.130:34563)
I20260811 00:24:22.991096 17060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:43017
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:24:23.036792 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39963
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:23.400683 17640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:23.401017 17640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:23.401129 17640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 00:24:23.405704 17389 heartbeater.cc:503] Master 127.16.71.190:39963 was elected leader, sending a full tablet report...
W20260811 00:24:23.409226 17640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:23.409350 17640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:23.409469 17640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:24:23.419936 17640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:39963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:23.421416 17640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:23.422788 17640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:23.438071 17650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:23.438048 17652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:23.438108 17654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:23.439019 17640 server_base.cc:1034] running on GCE node
I20260811 00:24:23.439721 17640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:23.440907 17640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:23.442179 17640 hybrid_clock.cc:648] HybridClock initialized: now 1786407863442130 us; error 36 us; skew 500 ppm
I20260811 00:24:23.442562 17640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:23 dist-test-slave-f7th krb5kdc[16992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407863, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:23.449324 17640 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:24:23.451371 17640 webserver.cc:460] Webserver started at http://127.16.71.131:41981/ using document root <none> and password file <none>
I20260811 00:24:23.452178 17640 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:23.452309 17640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:23.452732 17640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:23.456108 17640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "22a2920de4ac40cbaac4b029ca7965fe"
format_stamp: "Formatted at 2026-08-11 00:24:23 on dist-test-slave-f7th"
server_key: "31adebbafbb22725029707aed890eb56"
server_key_iv: "88948609e452c510c3cdcb04daab86e6"
server_key_version: "encryptionkey@0"
I20260811 00:24:23.457036 17640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "22a2920de4ac40cbaac4b029ca7965fe"
format_stamp: "Formatted at 2026-08-11 00:24:23 on dist-test-slave-f7th"
server_key: "31adebbafbb22725029707aed890eb56"
server_key_iv: "88948609e452c510c3cdcb04daab86e6"
server_key_version: "encryptionkey@0"
I20260811 00:24:23.463655 17640 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:23.468286 17662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:23.470188 17640 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:24:23.470419 17640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "22a2920de4ac40cbaac4b029ca7965fe"
format_stamp: "Formatted at 2026-08-11 00:24:23 on dist-test-slave-f7th"
server_key: "31adebbafbb22725029707aed890eb56"
server_key_iv: "88948609e452c510c3cdcb04daab86e6"
server_key_version: "encryptionkey@0"
I20260811 00:24:23.470681 17640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:23.491101 17640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:23.495908 17640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:23.496207 17640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:23.497339 17640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:23.498937 17640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:23.499025 17640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:23.499151 17640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:23.499203 17640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:23.533222 17640 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:43889
I20260811 00:24:23.533277 17877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:43889 every 8 connection(s)
I20260811 00:24:23.534662 17640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:24:23.542979 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17640
I20260811 00:24:23.543160 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:24:23.543691 16670 external_mini_cluster.cc:1428] Setting key 1b87c190d1980d0f28bd2d84f2bac17c
Aug 11 00:24:23 dist-test-slave-f7th krb5kdc[16992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407863, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:23.558202 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.537471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:50859 (local address 127.16.71.190:39963)
0811 00:24:23.538006 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.538014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.539371 (+ 1357us) server_negotiation.cc:407] Connection header received
0811 00:24:23.541505 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.541510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.541615 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.541737 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:23.543692 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.544806 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.546792 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.547293 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.547418 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.551519 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:23.551526 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:23.551529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:23.551570 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:23.554538 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:23.555485 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:23.555490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:23.555492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:23.555645 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:23.556129 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:23.556133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:23.556135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:23.556451 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:23.556652 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.557481 (+ 829us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.557763 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":221,"threads_started":1}
I20260811 00:24:23.559582 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.538092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:39963 (local address 127.16.71.131:50859)
0811 00:24:23.539039 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:23.539203 (+ 164us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:23.541135 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:23.542095 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:23.542105 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:23.542716 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:23.543460 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.543484 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.544998 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.545002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.546615 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.546631 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.547567 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.547587 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.547829 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.548670 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:23.548713 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:23.551220 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:23.554747 (+ 3527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:23.554756 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:23.554772 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:23.555321 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:23.555771 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:23.555775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:23.555777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:23.556006 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:23.556599 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:23.556608 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:23.557059 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 00:24:23.557371 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 00:24:23.557979 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":357,"threads_started":1}
I20260811 00:24:23.561141 17879 heartbeater.cc:347] Connected to a master server at 127.16.71.190:39963
I20260811 00:24:23.561542 17879 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:23.562440 17879 heartbeater.cc:511] Master 127.16.71.190:39963 requested a full tablet report, sending...
I20260811 00:24:23.564193 17060 ts_manager.cc:194] Registered new tserver with Master: 22a2920de4ac40cbaac4b029ca7965fe (127.16.71.131:43889)
I20260811 00:24:23.565128 17060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:50859
I20260811 00:24:23.573408 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:24:23.598484 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.578735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.16.71.190:39963)
0811 00:24:23.578984 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.578991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.579630 (+ 639us) server_negotiation.cc:407] Connection header received
0811 00:24:23.579830 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.579836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.579948 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.580064 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:23.582128 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.583281 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.585748 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.586564 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.586682 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.590199 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:23.590207 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:23.590212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:23.590263 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:23.594071 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:23.595474 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:23.595481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:23.595484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:23.595685 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:23.596401 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:23.596406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:23.596410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:23.596923 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:23.597191 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.597980 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.598203 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 00:24:23.604260 17060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:24:23.613832 17060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:24:23.660440 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.633194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.16.71.129:46767)
0811 00:24:23.652191 (+ 18997us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.652201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.652236 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:24:23.652347 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.652357 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.652666 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.652894 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:23.654076 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.655494 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.658129 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.659154 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.659315 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.659586 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.659840 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.660084 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18810,"thread_start_us":18640,"threads_started":1}
I20260811 00:24:23.661468 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.634560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.16.71.130:34563)
0811 00:24:23.649262 (+ 14702us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.649274 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.649317 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:24:23.649467 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.649479 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.649822 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.653217 (+ 3395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:23.654047 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.655841 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.658585 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.660039 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.660172 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.660365 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.660977 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.661150 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14466,"thread_start_us":14316,"threads_started":1}
I20260811 00:24:23.661523 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.634319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:34563 (local address 127.0.0.1:40422)
0811 00:24:23.642678 (+ 8359us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:23.642700 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:23.642853 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:23.652972 (+ 10119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:23.652982 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:23.653037 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:23.653798 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.653816 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.656003 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.656008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.658364 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.658381 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.660532 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.660539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.660751 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.660782 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:24:23.660882 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 00:24:23.661007 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8202,"mutex_wait_us":54,"spinlock_wait_cycles":8448,"thread_start_us":295,"threads_started":1}
I20260811 00:24:23.662139 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.632411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46767 (local address 127.0.0.1:44792)
0811 00:24:23.633356 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:23.633414 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:23.633650 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:23.652972 (+ 19322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:23.652984 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:23.653071 (+ 87us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:23.653810 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.653827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.655666 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.655672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.657929 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.657951 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.659363 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.659368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.659570 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.659634 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 00:24:23.659779 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 00:24:23.659993 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":126,"spinlock_wait_cycles":9088,"thread_start_us":1004,"threads_started":1}
I20260811 00:24:23.663348 17906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.632874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43889 (local address 127.0.0.1:57148)
0811 00:24:23.634943 (+ 2069us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:23.634964 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:23.635101 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:23.654627 (+ 19526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:23.654632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:23.654664 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:23.655020 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.655033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.656881 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.656886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.659036 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.659053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.661452 (+ 2399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.661459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.662759 (+ 1300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.662787 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:24:23.662880 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:24:23.663022 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1923,"thread_start_us":9446,"threads_started":1}
I20260811 00:24:23.663477 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.633068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57148 (local address 127.16.71.131:43889)
0811 00:24:23.653974 (+ 20906us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.653982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.654018 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:24:23.654113 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.654121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.654381 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.654601 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:23.655220 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.656723 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.659236 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.660935 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.661065 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.661269 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.662965 (+ 1696us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.663153 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20707,"thread_start_us":20526,"threads_started":1}
I20260811 00:24:23.665934 17510 tablet_service.cc:1467] Processing CreateTablet for tablet c04ba1f3872c4536b637bbdbd1fe4edf (DEFAULT_TABLE table=test-table [id=cb9681a968414e94ad3b4cb2466069f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:23.665934 17280 tablet_service.cc:1467] Processing CreateTablet for tablet c04ba1f3872c4536b637bbdbd1fe4edf (DEFAULT_TABLE table=test-table [id=cb9681a968414e94ad3b4cb2466069f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:23.666590 17759 tablet_service.cc:1467] Processing CreateTablet for tablet c04ba1f3872c4536b637bbdbd1fe4edf (DEFAULT_TABLE table=test-table [id=cb9681a968414e94ad3b4cb2466069f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:23.667579 17280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04ba1f3872c4536b637bbdbd1fe4edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:23.667577 17510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04ba1f3872c4536b637bbdbd1fe4edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:23.668227 17759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04ba1f3872c4536b637bbdbd1fe4edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:23.680428 17908 tablet_bootstrap.cc:492] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Bootstrap starting.
I20260811 00:24:23.683262 17909 tablet_bootstrap.cc:492] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Bootstrap starting.
I20260811 00:24:23.686627 17908 tablet_bootstrap.cc:654] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:23.687314 17909 tablet_bootstrap.cc:654] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:23.688385 17908 log.cc:824] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:23.688552 17909 log.cc:824] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:23.698451 17908 tablet_bootstrap.cc:492] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: No bootstrap required, opened a new log
I20260811 00:24:23.699031 17908 ts_tablet_manager.cc:1397] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.005s
I20260811 00:24:23.701495 17910 tablet_bootstrap.cc:492] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Bootstrap starting.
I20260811 00:24:23.705447 17910 tablet_bootstrap.cc:654] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:23.706944 17910 log.cc:824] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:23.709022 17909 tablet_bootstrap.cc:492] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: No bootstrap required, opened a new log
I20260811 00:24:23.709395 17909 ts_tablet_manager.cc:1397] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.016s
I20260811 00:24:23.710125 17910 tablet_bootstrap.cc:492] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: No bootstrap required, opened a new log
I20260811 00:24:23.710439 17910 ts_tablet_manager.cc:1397] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:24:23.713840 17908 raft_consensus.cc:348] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.714368 17908 raft_consensus.cc:374] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:23.714490 17908 raft_consensus.cc:729] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a2920de4ac40cbaac4b029ca7965fe, State: Initialized, Role: FOLLOWER
I20260811 00:24:23.715232 17908 consensus_queue.cc:260] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.716534 17879 heartbeater.cc:503] Master 127.16.71.190:39963 was elected leader, sending a full tablet report...
I20260811 00:24:23.717396 17908 ts_tablet_manager.cc:1428] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Time spent starting tablet: real 0.018s user 0.012s sys 0.003s
I20260811 00:24:23.718959 17910 raft_consensus.cc:348] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.719447 17910 raft_consensus.cc:374] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:23.719563 17910 raft_consensus.cc:729] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 627132988d3c4588ae064a6574cddf6f, State: Initialized, Role: FOLLOWER
I20260811 00:24:23.720252 17910 consensus_queue.cc:260] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.721141 17909 raft_consensus.cc:348] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.721637 17909 raft_consensus.cc:374] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:23.721753 17909 raft_consensus.cc:729] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 458d95bd19b24224b799128b0f44c90c, State: Initialized, Role: FOLLOWER
I20260811 00:24:23.722226 17910 ts_tablet_manager.cc:1428] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 00:24:23.722661 17909 consensus_queue.cc:260] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.726462 17909 ts_tablet_manager.cc:1428] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
I20260811 00:24:23.727017 17630 heartbeater.cc:503] Master 127.16.71.190:39963 was elected leader, sending a full tablet report...
W20260811 00:24:23.786567 17881 tablet.cc:2369] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:23.847288 17923 raft_consensus.cc:482] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:24:23.847636 17923 raft_consensus.cc:504] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.849644 17923 leader_election.cc:288] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 627132988d3c4588ae064a6574cddf6f (127.16.71.129:46767), 22a2920de4ac40cbaac4b029ca7965fe (127.16.71.131:43889)
I20260811 00:24:23.856779 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.850278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:54815 (local address 127.16.71.131:43889)
0811 00:24:23.850473 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.850479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.851062 (+ 583us) server_negotiation.cc:407] Connection header received
0811 00:24:23.851299 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.851303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.851369 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.851445 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:23.852183 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.853001 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.854927 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.855746 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.855921 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.856005 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.856338 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.856476 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 00:24:23.856877 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.850046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:47083 (local address 127.16.71.129:46767)
0811 00:24:23.850340 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.850348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.851173 (+ 825us) server_negotiation.cc:407] Connection header received
0811 00:24:23.851358 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.851362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.851427 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.851514 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:23.852221 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.853000 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.855012 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.855855 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.856031 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.856112 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.856432 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.856584 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 00:24:23.856930 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.849970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:43889 (local address 127.16.71.130:54815)
0811 00:24:23.850940 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:23.850956 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:23.851157 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:23.851473 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:23.851478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:23.851510 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:23.852042 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.852053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.853131 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.853137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.854738 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.854755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.855991 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.855998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.856165 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.856195 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:24:23.856286 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:24:23.856434 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":2,"spinlock_wait_cycles":27392,"thread_start_us":567,"threads_started":1}
I20260811 00:24:23.857388 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.850054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:46767 (local address 127.16.71.130:47083)
0811 00:24:23.851066 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:23.851081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:23.851199 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:23.851546 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:23.851551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:23.851580 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:23.852049 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.852062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.853138 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.853142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.854745 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:23.854759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.856101 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.856108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:23.856269 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.856294 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 00:24:23.856377 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 00:24:23.856517 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":23,"spinlock_wait_cycles":768,"thread_start_us":454,"threads_started":1}
I20260811 00:24:23.858130 17798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ba1f3872c4536b637bbdbd1fe4edf" candidate_uuid: "458d95bd19b24224b799128b0f44c90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a2920de4ac40cbaac4b029ca7965fe" is_pre_election: true
I20260811 00:24:23.858423 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ba1f3872c4536b637bbdbd1fe4edf" candidate_uuid: "458d95bd19b24224b799128b0f44c90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "627132988d3c4588ae064a6574cddf6f" is_pre_election: true
I20260811 00:24:23.858812 17798 raft_consensus.cc:2454] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 458d95bd19b24224b799128b0f44c90c in term 0.
I20260811 00:24:23.858985 17312 raft_consensus.cc:2454] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 458d95bd19b24224b799128b0f44c90c in term 0.
I20260811 00:24:23.859637 17432 leader_election.cc:302] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a2920de4ac40cbaac4b029ca7965fe, 458d95bd19b24224b799128b0f44c90c; no voters:
I20260811 00:24:23.860072 17923 raft_consensus.cc:2781] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:24:23.860260 17923 raft_consensus.cc:482] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:24:23.860383 17923 raft_consensus.cc:3037] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:23.862905 17923 raft_consensus.cc:504] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
I20260811 00:24:23.863658 17923 leader_election.cc:288] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [CANDIDATE]: Term 1 election: Requested vote from peers 627132988d3c4588ae064a6574cddf6f (127.16.71.129:46767), 22a2920de4ac40cbaac4b029ca7965fe (127.16.71.131:43889)
I20260811 00:24:23.864302 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ba1f3872c4536b637bbdbd1fe4edf" candidate_uuid: "458d95bd19b24224b799128b0f44c90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "627132988d3c4588ae064a6574cddf6f"
I20260811 00:24:23.864377 17798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ba1f3872c4536b637bbdbd1fe4edf" candidate_uuid: "458d95bd19b24224b799128b0f44c90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a2920de4ac40cbaac4b029ca7965fe"
I20260811 00:24:23.864567 17312 raft_consensus.cc:3037] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:23.864594 17798 raft_consensus.cc:3037] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:23.867079 17798 raft_consensus.cc:2454] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 458d95bd19b24224b799128b0f44c90c in term 1.
I20260811 00:24:23.867077 17312 raft_consensus.cc:2454] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 458d95bd19b24224b799128b0f44c90c in term 1.
I20260811 00:24:23.867672 17430 leader_election.cc:302] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 458d95bd19b24224b799128b0f44c90c, 627132988d3c4588ae064a6574cddf6f; no voters:
I20260811 00:24:23.868098 17923 raft_consensus.cc:2781] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:23.868723 17923 raft_consensus.cc:686] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 1 LEADER]: Becoming Leader. State: Replica: 458d95bd19b24224b799128b0f44c90c, State: Running, Role: LEADER
I20260811 00:24:23.869268 17923 consensus_queue.cc:237] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } }
W20260811 00:24:23.874233 17390 tablet.cc:2369] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:23.875135 17060 catalog_manager.cc:5495] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c reported cstate change: term changed from 0 to 1, leader changed from <none> to 458d95bd19b24224b799128b0f44c90c (127.16.71.130). New cstate: current_term: 1 leader_uuid: "458d95bd19b24224b799128b0f44c90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "458d95bd19b24224b799128b0f44c90c" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 34563 } health_report { overall_health: HEALTHY } } }
I20260811 00:24:23.902356 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.895818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.16.71.130:34563)
0811 00:24:23.896020 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.896025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.896050 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:24:23.896253 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.896257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.896356 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.896514 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:24:23.896990 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.897755 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.899732 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.900197 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.900324 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.901135 (+ 811us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:24:23.901719 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:24:23.901867 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.902072 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.902120 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:24:23.911583 17312 raft_consensus.cc:1261] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 1 FOLLOWER]: Refusing update from remote peer 458d95bd19b24224b799128b0f44c90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:23.911583 17798 raft_consensus.cc:1261] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 1 FOLLOWER]: Refusing update from remote peer 458d95bd19b24224b799128b0f44c90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:23.912807 17927 consensus_queue.cc:1035] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "627132988d3c4588ae064a6574cddf6f" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 46767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:23.913244 17923 consensus_queue.cc:1035] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a2920de4ac40cbaac4b029ca7965fe" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 43889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:23.926497 17938 mvcc.cc:205] Tried to move back new op lower bound from 7317126610570498048 to 7317126610412777472. Current Snapshot: MvccSnapshot[applied={T|T < 7317126610570498048}]
I20260811 00:24:23.926508 17931 mvcc.cc:205] Tried to move back new op lower bound from 7317126610570498048 to 7317126610412777472. Current Snapshot: MvccSnapshot[applied={T|T < 7317126610570498048}]
I20260811 00:24:23.941807 17056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:23.942117 17056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:23.946504 17056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0aafdfa17fbc45a085d0ecdc3c5da7bb: Sending DeleteTablet for 3 replicas of tablet c04ba1f3872c4536b637bbdbd1fe4edf
I20260811 00:24:23.948040 17759 tablet_service.cc:1514] Processing DeleteTablet for tablet c04ba1f3872c4536b637bbdbd1fe4edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:23 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:57148
I20260811 00:24:23.948357 17510 tablet_service.cc:1514] Processing DeleteTablet for tablet c04ba1f3872c4536b637bbdbd1fe4edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:23 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:40422
I20260811 00:24:23.949162 17280 tablet_service.cc:1514] Processing DeleteTablet for tablet c04ba1f3872c4536b637bbdbd1fe4edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:23 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:44792
I20260811 00:24:23.949388 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17169
I20260811 00:24:23.957070 17951 tablet_replica.cc:330] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: stopping tablet replica
I20260811 00:24:23.957746 17951 raft_consensus.cc:2229] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:24:23.958840 17951 raft_consensus.cc:2258] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:23.961740 17951 ts_tablet_manager.cc:1905] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:23.966857 17949 tablet_replica.cc:330] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: stopping tablet replica
I20260811 00:24:23.967839 17952 tablet_replica.cc:330] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: stopping tablet replica
I20260811 00:24:23.968441 17952 raft_consensus.cc:2229] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:23.969147 17949 raft_consensus.cc:2229] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:23.970703 17951 ts_tablet_manager.cc:1918] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:23.971001 17951 log.cc:1196] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/c04ba1f3872c4536b637bbdbd1fe4edf
I20260811 00:24:23.971807 17952 raft_consensus.cc:2258] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:23.972115 17951 ts_tablet_manager.cc:1939] T c04ba1f3872c4536b637bbdbd1fe4edf P 458d95bd19b24224b799128b0f44c90c: Deleting consensus metadata
I20260811 00:24:23.972395 17949 raft_consensus.cc:2258] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:23.974699 17037 catalog_manager.cc:4841] TS 458d95bd19b24224b799128b0f44c90c (127.16.71.130:34563): tablet c04ba1f3872c4536b637bbdbd1fe4edf (table test-table [id=cb9681a968414e94ad3b4cb2466069f5]) successfully deleted
I20260811 00:24:23.977479 17952 ts_tablet_manager.cc:1905] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:23.983310 17949 ts_tablet_manager.cc:1905] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:24:23 dist-test-slave-f7th krb5kdc[16992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:24:23.985051 17952 ts_tablet_manager.cc:1918] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:23.985330 17952 log.cc:1196] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/c04ba1f3872c4536b637bbdbd1fe4edf
I20260811 00:24:23.985986 17952 ts_tablet_manager.cc:1939] T c04ba1f3872c4536b637bbdbd1fe4edf P 627132988d3c4588ae064a6574cddf6f: Deleting consensus metadata
I20260811 00:24:23.988075 17037 catalog_manager.cc:4841] TS 627132988d3c4588ae064a6574cddf6f (127.16.71.129:46767): tablet c04ba1f3872c4536b637bbdbd1fe4edf (table test-table [id=cb9681a968414e94ad3b4cb2466069f5]) successfully deleted
I20260811 00:24:23.992480 17949 ts_tablet_manager.cc:1918] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:23.992735 17949 log.cc:1196] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/c04ba1f3872c4536b637bbdbd1fe4edf
I20260811 00:24:23.993319 17949 ts_tablet_manager.cc:1939] T c04ba1f3872c4536b637bbdbd1fe4edf P 22a2920de4ac40cbaac4b029ca7965fe: Deleting consensus metadata
I20260811 00:24:23.995566 17040 catalog_manager.cc:4841] TS 22a2920de4ac40cbaac4b029ca7965fe (127.16.71.131:43889): tablet c04ba1f3872c4536b637bbdbd1fe4edf (table test-table [id=cb9681a968414e94ad3b4cb2466069f5]) successfully deleted
I20260811 00:24:23.997123 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:23.950878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.16.71.129:46767)
0811 00:24:23.951079 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:23.951086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:23.956694 (+ 5608us) server_negotiation.cc:407] Connection header received
0811 00:24:23.957824 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:23.957830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:23.957939 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:23.958019 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:23.961884 (+ 3865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.963134 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.969530 (+ 6396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:23.970233 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:23.970383 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:23.990263 (+ 19880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:23.990278 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:23.990294 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:23.990336 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:23.993363 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:23.994196 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:23.994205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:23.994207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:23.994363 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:23.995132 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:23.995137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:23.995140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:23.995686 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:23.996064 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:23.996511 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 00:24:23.996799 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:24:23.997926 17377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:24.092480 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17169
I20260811 00:24:24.130718 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17398
Aug 11 00:24:24 dist-test-slave-f7th krb5kdc[16992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:24:24.151356 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:24.131714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.16.71.130:34563)
0811 00:24:24.131940 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:24.131947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:24.131970 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:24:24.132038 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:24.132047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:24.132138 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:24.132251 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:24.133864 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:24.135194 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:24.136739 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:24.137450 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:24.137616 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:24.142792 (+ 5176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:24.142815 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:24.142843 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:24.142899 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:24.147479 (+ 4580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:24.148558 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:24.148569 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:24.148573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:24.148818 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:24.149306 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:24.149312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:24.149315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:24.149857 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:24.150211 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:24.150669 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 00:24:24.150943 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:24:24.152410 17622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:24.271797 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17398
I20260811 00:24:24.301928 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17640
Aug 11 00:24:24 dist-test-slave-f7th krb5kdc[16992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:24:24.319051 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:24.302718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.16.71.131:43889)
0811 00:24:24.302903 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:24.302907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:24.302931 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:24:24.303124 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:24.303128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:24.303197 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:24.303284 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:24.304751 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:24.305511 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:24.306997 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:24.307425 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:24.307540 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:24.312514 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:24.312536 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:24.312554 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:24.312605 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:24.315666 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:24.316560 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:24.316569 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:24.316572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:24.316984 (+ 412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:24.317490 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:24.317495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:24.317497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:24.317939 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:24.318186 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:24.318456 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 00:24:24.318646 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:24:24.319710 17870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:24.410238 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17640
I20260811 00:24:24.436185 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17013
I20260811 00:24:24.437628 17116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:24.556454 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17013
2026-08-11T00:24:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5068 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T00:24:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:24Z Disabled control of system clock
I20260811 00:24:24.661041 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:32883
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:32883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:24:25.041533 17968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:25.042059 17968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:25.042203 17968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:25.050123 17968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:25.050273 17968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:25.050313 17968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:25.050380 17968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:25.050424 17968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:25.060262 17968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:32883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:32883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:25.061748 17968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:25.063716 17968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:25.077778 17982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:25.077847 17977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:25.077816 17980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:25.078631 17968 server_base.cc:1034] running on GCE node
I20260811 00:24:25.079573 17968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:25.081300 17968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:25.082533 17968 hybrid_clock.cc:648] HybridClock initialized: now 1786407865082474 us; error 50 us; skew 500 ppm
I20260811 00:24:25.082913 17968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:25.085551 17968 webserver.cc:460] Webserver started at http://127.16.71.190:46337/ using document root <none> and password file <none>
I20260811 00:24:25.086334 17968 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:25.086455 17968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:25.086838 17968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:25.090194 17968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "c833f95d6cae44ca8040170ac37be38f"
format_stamp: "Formatted at 2026-08-11 00:24:25 on dist-test-slave-f7th"
server_key: "5abe3ee16ed2d9d749dbbaa9c72ace17"
server_key_iv: "ea33ad2d19f5007d4caa7abb42aea216"
server_key_version: "encryptionkey@0"
I20260811 00:24:25.091125 17968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "c833f95d6cae44ca8040170ac37be38f"
format_stamp: "Formatted at 2026-08-11 00:24:25 on dist-test-slave-f7th"
server_key: "5abe3ee16ed2d9d749dbbaa9c72ace17"
server_key_iv: "ea33ad2d19f5007d4caa7abb42aea216"
server_key_version: "encryptionkey@0"
I20260811 00:24:25.097270 17968 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:24:25.101295 17994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:25.102864 17968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:24:25.103080 17968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "c833f95d6cae44ca8040170ac37be38f"
format_stamp: "Formatted at 2026-08-11 00:24:25 on dist-test-slave-f7th"
server_key: "5abe3ee16ed2d9d749dbbaa9c72ace17"
server_key_iv: "ea33ad2d19f5007d4caa7abb42aea216"
server_key_version: "encryptionkey@0"
I20260811 00:24:25.103327 17968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:25.129786 17968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:25.130781 17968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:25.131096 17968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:25.150693 17968 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:32883
I20260811 00:24:25.150703 18076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:32883 every 8 connection(s)
I20260811 00:24:25.152191 17968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:25.156483 18078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:25.157099 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 17968
I20260811 00:24:25.157260 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:25.157714 16670 external_mini_cluster.cc:1428] Setting key 709414cb44f8f3fd63f19083ed00e43d
I20260811 00:24:25.172497 18078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f: Bootstrap starting.
I20260811 00:24:25.175201 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:25.159936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.16.71.190:32883)
0811 00:24:25.161216 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:25.161233 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:25.161302 (+ 69us) server_negotiation.cc:407] Connection header received
0811 00:24:25.163504 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:25.163544 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:25.164328 (+ 784us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:25.164905 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:25.165637 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:25.167624 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:25.169322 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:25.170569 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:25.170793 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:25.172027 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:25.172039 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:25.172062 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:25.172866 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:25.172979 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:25.173010 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:25.173143 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:25.173369 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 00:24:25.173753 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":385,"threads_started":1}
I20260811 00:24:25.176839 18078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:25.178256 18078 log.cc:824] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:25.181339 18078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f: No bootstrap required, opened a new log
I20260811 00:24:25.189134 18078 raft_consensus.cc:348] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c833f95d6cae44ca8040170ac37be38f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32883 } }
I20260811 00:24:25.189555 18078 raft_consensus.cc:374] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:25.189658 18078 raft_consensus.cc:729] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c833f95d6cae44ca8040170ac37be38f, State: Initialized, Role: FOLLOWER
I20260811 00:24:25.190290 18078 consensus_queue.cc:260] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c833f95d6cae44ca8040170ac37be38f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32883 } }
I20260811 00:24:25.190616 18078 raft_consensus.cc:388] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:25.190783 18078 raft_consensus.cc:482] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:25.190989 18078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:25.193636 18078 raft_consensus.cc:504] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c833f95d6cae44ca8040170ac37be38f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32883 } }
I20260811 00:24:25.194185 18078 leader_election.cc:302] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c833f95d6cae44ca8040170ac37be38f; no voters:
I20260811 00:24:25.194796 18078 leader_election.cc:288] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:25.194967 18090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:25.195686 18090 raft_consensus.cc:686] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [term 1 LEADER]: Becoming Leader. State: Replica: c833f95d6cae44ca8040170ac37be38f, State: Running, Role: LEADER
I20260811 00:24:25.196255 18090 consensus_queue.cc:237] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c833f95d6cae44ca8040170ac37be38f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32883 } }
I20260811 00:24:25.197100 18078 sys_catalog.cc:564] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:25.199765 18093 sys_catalog.cc:455] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c833f95d6cae44ca8040170ac37be38f. Latest consensus state: current_term: 1 leader_uuid: "c833f95d6cae44ca8040170ac37be38f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c833f95d6cae44ca8040170ac37be38f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32883 } } }
I20260811 00:24:25.200255 18093 sys_catalog.cc:458] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:25.205922 18091 sys_catalog.cc:455] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c833f95d6cae44ca8040170ac37be38f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c833f95d6cae44ca8040170ac37be38f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 32883 } } }
I20260811 00:24:25.206425 18091 sys_catalog.cc:458] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:25.208786 18100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:25.213492 18100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:25.224682 18100 catalog_manager.cc:1347] Generated new cluster ID: 65ad35b092674f028d845218af88420f
I20260811 00:24:25.224831 18100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:25.244081 18100 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:25.245549 18100 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:25.256466 18100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f: Generated new TSK 0
I20260811 00:24:25.257513 18100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:25.265885 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32883
--builtin_ntp_servers=127.16.71.148:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:24:25.643630 18114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:25.643978 18114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:25.644074 18114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:25.651638 18114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:25.651780 18114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:25.651893 18114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:24:25.662185 18114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:25.663570 18114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:25.665541 18114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:25.680660 18121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:25.680750 18123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:25.680845 18127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:25.681463 18114 server_base.cc:1034] running on GCE node
I20260811 00:24:25.682183 18114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:25.683229 18114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:25.684443 18114 hybrid_clock.cc:648] HybridClock initialized: now 1786407865684394 us; error 37 us; skew 500 ppm
I20260811 00:24:25.684809 18114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:25.687358 18114 webserver.cc:460] Webserver started at http://127.16.71.129:39593/ using document root <none> and password file <none>
I20260811 00:24:25.688126 18114 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:25.688228 18114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:25.688551 18114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:25.691773 18114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "767bc9f5ff684cc2baf572863bd89c95"
format_stamp: "Formatted at 2026-08-11 00:24:25 on dist-test-slave-f7th"
server_key: "68457d2ee4ca8e0b385d761617d39607"
server_key_iv: "edc716af27943bdf3e7a7a1a7e46f156"
server_key_version: "encryptionkey@0"
I20260811 00:24:25.692781 18114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "767bc9f5ff684cc2baf572863bd89c95"
format_stamp: "Formatted at 2026-08-11 00:24:25 on dist-test-slave-f7th"
server_key: "68457d2ee4ca8e0b385d761617d39607"
server_key_iv: "edc716af27943bdf3e7a7a1a7e46f156"
server_key_version: "encryptionkey@0"
I20260811 00:24:25.698830 18114 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:25.702993 18137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:25.704789 18114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:24:25.704983 18114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "767bc9f5ff684cc2baf572863bd89c95"
format_stamp: "Formatted at 2026-08-11 00:24:25 on dist-test-slave-f7th"
server_key: "68457d2ee4ca8e0b385d761617d39607"
server_key_iv: "edc716af27943bdf3e7a7a1a7e46f156"
server_key_version: "encryptionkey@0"
I20260811 00:24:25.705220 18114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:25.726698 18114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:25.727651 18114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:25.727962 18114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:25.729199 18114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:25.730893 18114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:25.730993 18114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:25.731109 18114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:25.731200 18114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:25.764091 18114 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:33237
I20260811 00:24:25.764173 18341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:33237 every 8 connection(s)
I20260811 00:24:25.765573 18114 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:24:25.769563 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18114
I20260811 00:24:25.769757 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:24:25.770303 16670 external_mini_cluster.cc:1428] Setting key 426f5704cee0a42112775c3c3df9bc2d
I20260811 00:24:25.773404 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32883
--builtin_ntp_servers=127.16.71.148:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:24:25.780185 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:25.768405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:52825 (local address 127.16.71.190:32883)
0811 00:24:25.768963 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:25.768968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:25.770356 (+ 1388us) server_negotiation.cc:407] Connection header received
0811 00:24:25.772479 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:25.772485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:25.772574 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:25.772719 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:25.773949 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:25.774980 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:25.776950 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:25.777510 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:25.777659 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:25.779270 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:25.779277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:25.779281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:25.779309 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:25.779361 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:25.779376 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:25.779471 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:25.779707 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 00:24:25.779821 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":309,"threads_started":1}
I20260811 00:24:25.781332 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:25.768966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:32883 (local address 127.16.71.129:52825)
0811 00:24:25.770127 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:25.770190 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:25.772079 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:25.772951 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:25.772961 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:25.773062 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:25.773758 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:25.773781 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:25.775176 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:25.775180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:25.776739 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:25.776759 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:25.777807 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:25.777814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:25.778098 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:25.778842 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:25.778845 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:25.778893 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:25.779031 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:25.779484 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:25.779495 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:25.779502 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:24:25.779672 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 00:24:25.780084 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":366,"threads_started":1}
I20260811 00:24:25.783142 18342 heartbeater.cc:347] Connected to a master server at 127.16.71.190:32883
I20260811 00:24:25.783478 18342 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:25.784317 18342 heartbeater.cc:511] Master 127.16.71.190:32883 requested a full tablet report, sending...
I20260811 00:24:25.786784 18022 ts_manager.cc:194] Registered new tserver with Master: 767bc9f5ff684cc2baf572863bd89c95 (127.16.71.129:33237)
I20260811 00:24:25.788792 18022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.129:52825
W20260811 00:24:26.142504 18349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:26.142818 18349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:26.142908 18349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:26.150615 18349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:26.150732 18349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:26.150869 18349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:24:26.160429 18349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:26.161818 18349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:26.163656 18349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:26.177902 18356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:26.177942 18357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:26.177861 18363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:26.178875 18349 server_base.cc:1034] running on GCE node
I20260811 00:24:26.179589 18349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:26.180747 18349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:26.181980 18349 hybrid_clock.cc:648] HybridClock initialized: now 1786407866181905 us; error 63 us; skew 500 ppm
I20260811 00:24:26.182323 18349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:26.184960 18349 webserver.cc:460] Webserver started at http://127.16.71.130:44385/ using document root <none> and password file <none>
I20260811 00:24:26.185709 18349 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:26.185824 18349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:26.186153 18349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:26.189477 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "292bee60b3bd460bb8279496a51b75ee"
format_stamp: "Formatted at 2026-08-11 00:24:26 on dist-test-slave-f7th"
server_key: "2eb11e54878f446735f6b7d14e25ddf6"
server_key_iv: "c76e38cd691e10b883cf16ae262e9a82"
server_key_version: "encryptionkey@0"
I20260811 00:24:26.190377 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "292bee60b3bd460bb8279496a51b75ee"
format_stamp: "Formatted at 2026-08-11 00:24:26 on dist-test-slave-f7th"
server_key: "2eb11e54878f446735f6b7d14e25ddf6"
server_key_iv: "c76e38cd691e10b883cf16ae262e9a82"
server_key_version: "encryptionkey@0"
I20260811 00:24:26.196398 18349 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:24:26.200301 18372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:26.201956 18349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:24:26.202173 18349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "292bee60b3bd460bb8279496a51b75ee"
format_stamp: "Formatted at 2026-08-11 00:24:26 on dist-test-slave-f7th"
server_key: "2eb11e54878f446735f6b7d14e25ddf6"
server_key_iv: "c76e38cd691e10b883cf16ae262e9a82"
server_key_version: "encryptionkey@0"
I20260811 00:24:26.202432 18349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:26.233863 18349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:26.234885 18349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:26.235262 18349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:26.236406 18349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:26.238204 18349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:26.238320 18349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:26.238447 18349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:26.238523 18349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:26.280452 18349 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:36963
I20260811 00:24:26.280501 18556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:36963 every 8 connection(s)
I20260811 00:24:26.282310 18349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:24:26.291491 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18349
I20260811 00:24:26.291676 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:24:26.292233 16670 external_mini_cluster.cc:1428] Setting key 049b347eada56e4d1fdc9dfb640ff7dc
I20260811 00:24:26.295559 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32883
--builtin_ntp_servers=127.16.71.148:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:24:26.302780 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.290860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:43817 (local address 127.16.71.190:32883)
0811 00:24:26.291385 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:26.291391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:26.292884 (+ 1493us) server_negotiation.cc:407] Connection header received
0811 00:24:26.295202 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:26.295209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:26.295300 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:26.295432 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:26.296596 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.297733 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.299623 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.300177 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.300311 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.301907 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:26.301913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:26.301917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:26.301945 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:26.301997 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:26.302015 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:26.302103 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:26.302331 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 00:24:26.302444 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":252,"threads_started":1}
I20260811 00:24:26.304080 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.291518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:32883 (local address 127.16.71.130:43817)
0811 00:24:26.292702 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:26.292758 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:26.294896 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:26.295663 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:26.295673 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:26.295776 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:26.296399 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.296418 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.297916 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.297922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.299449 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.299467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.300401 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.300406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.300682 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.301518 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:26.301520 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:26.301566 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:26.301660 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:26.302120 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:26.302133 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:26.302140 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:24:26.302304 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 00:24:26.302694 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":377,"threads_started":1}
I20260811 00:24:26.305622 18557 heartbeater.cc:347] Connected to a master server at 127.16.71.190:32883
I20260811 00:24:26.305944 18557 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:26.306872 18557 heartbeater.cc:511] Master 127.16.71.190:32883 requested a full tablet report, sending...
I20260811 00:24:26.308609 18022 ts_manager.cc:194] Registered new tserver with Master: 292bee60b3bd460bb8279496a51b75ee (127.16.71.130:36963)
I20260811 00:24:26.309528 18022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.130:43817
W20260811 00:24:26.667002 18565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:26.667364 18565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:26.667487 18565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:26.675141 18565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:26.675314 18565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:26.675460 18565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:24:26.685518 18565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:32883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:26.687002 18565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:26.689109 18565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:26.703474 18578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:26.703563 18579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:26.703529 18584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:26.704600 18565 server_base.cc:1034] running on GCE node
I20260811 00:24:26.705314 18565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:26.706425 18565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:26.707674 18565 hybrid_clock.cc:648] HybridClock initialized: now 1786407866707614 us; error 52 us; skew 500 ppm
I20260811 00:24:26.708094 18565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:26.720876 18565 webserver.cc:460] Webserver started at http://127.16.71.131:44655/ using document root <none> and password file <none>
I20260811 00:24:26.721737 18565 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:26.721858 18565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:26.722203 18565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:26.725525 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "2304a6d192e1475ca74e4e1deddf6f08"
format_stamp: "Formatted at 2026-08-11 00:24:26 on dist-test-slave-f7th"
server_key: "962b508fe23eba5791631b7b69a733ba"
server_key_iv: "7c318dc44107142cc82729afa62317b8"
server_key_version: "encryptionkey@0"
I20260811 00:24:26.726454 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "2304a6d192e1475ca74e4e1deddf6f08"
format_stamp: "Formatted at 2026-08-11 00:24:26 on dist-test-slave-f7th"
server_key: "962b508fe23eba5791631b7b69a733ba"
server_key_iv: "7c318dc44107142cc82729afa62317b8"
server_key_version: "encryptionkey@0"
I20260811 00:24:26.732649 18565 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:24:26.737075 18591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:26.738889 18565 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:24:26.739131 18565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "2304a6d192e1475ca74e4e1deddf6f08"
format_stamp: "Formatted at 2026-08-11 00:24:26 on dist-test-slave-f7th"
server_key: "962b508fe23eba5791631b7b69a733ba"
server_key_iv: "7c318dc44107142cc82729afa62317b8"
server_key_version: "encryptionkey@0"
I20260811 00:24:26.739392 18565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:26.760162 18565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:26.761224 18565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:26.761538 18565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:26.762910 18565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:26.764686 18565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:26.764806 18565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:26.764923 18565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:26.765019 18565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:26.793605 18342 heartbeater.cc:503] Master 127.16.71.190:32883 was elected leader, sending a full tablet report...
I20260811 00:24:26.799777 18565 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:42173
I20260811 00:24:26.799865 18793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:42173 every 8 connection(s)
I20260811 00:24:26.801275 18565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:24:26.802003 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18565
I20260811 00:24:26.802218 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:24:26.802796 16670 external_mini_cluster.cc:1428] Setting key bc017aa5c814907dbb493151438d1990
I20260811 00:24:26.815757 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.804349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:42855 (local address 127.16.71.190:32883)
0811 00:24:26.804894 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:26.804901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:26.806997 (+ 2096us) server_negotiation.cc:407] Connection header received
0811 00:24:26.809073 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:26.809077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:26.809152 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:26.809240 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:26.810380 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.811197 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.812968 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.813394 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.813509 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.814938 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:26.814943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:26.814947 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:26.814970 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:26.815020 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:26.815032 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:26.815106 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:26.815355 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 00:24:26.815451 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":255,"threads_started":1}
I20260811 00:24:26.817018 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.805537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:32883 (local address 127.16.71.131:42855)
0811 00:24:26.806795 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:26.806876 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:26.808761 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:26.809458 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:26.809467 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:26.809570 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:26.810188 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.810212 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.811403 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.811409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.812790 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.812805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.813645 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.813651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.813901 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.814582 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:26.814584 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:26.814629 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:26.814723 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:26.815134 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:26.815144 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:26.815151 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:24:26.815313 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 00:24:26.815665 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":363,"threads_started":1}
I20260811 00:24:26.818619 18794 heartbeater.cc:347] Connected to a master server at 127.16.71.190:32883
I20260811 00:24:26.818974 18794 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:26.819871 18794 heartbeater.cc:511] Master 127.16.71.190:32883 requested a full tablet report, sending...
I20260811 00:24:26.821458 18022 ts_manager.cc:194] Registered new tserver with Master: 2304a6d192e1475ca74e4e1deddf6f08 (127.16.71.131:42173)
I20260811 00:24:26.822305 18022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.131:42855
I20260811 00:24:26.831737 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:24:26.842962 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.836386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.16.71.190:32883)
0811 00:24:26.836688 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:26.836696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:26.836950 (+ 254us) server_negotiation.cc:407] Connection header received
0811 00:24:26.837144 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:26.837149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:26.837217 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:26.837302 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:26.837894 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.838632 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.840147 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.840584 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.840755 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.842152 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:26.842158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:26.842161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:26.842184 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:26.842220 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:26.842233 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:26.842405 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:26.842597 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 00:24:26.842663 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 00:24:26.847793 18022 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:24:26.855123 18022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:24:26.893194 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.869565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:42173 (local address 127.0.0.1:34814)
0811 00:24:26.883247 (+ 13682us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:26.883295 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:26.883584 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:26.884731 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:26.884740 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:26.884783 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:26.885395 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.885412 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.887669 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.887674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.890336 (+ 2662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.890352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.892341 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.892348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.892502 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.892532 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:24:26.892589 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 00:24:26.892823 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13443,"mutex_wait_us":106,"thread_start_us":13282,"threads_started":1}
I20260811 00:24:26.893239 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.870019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.16.71.131:42173)
0811 00:24:26.881738 (+ 11719us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:26.881751 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:26.883453 (+ 1702us) server_negotiation.cc:407] Connection header received
0811 00:24:26.883908 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:26.883917 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:26.884162 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:26.884371 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:26.885692 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.887512 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.890616 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.892116 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.892282 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.892578 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:26.892713 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 00:24:26.892891 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11319,"thread_start_us":11767,"threads_started":1}
I20260811 00:24:26.893211 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.869565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.16.71.129:33237)
0811 00:24:26.882664 (+ 13099us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:26.882674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:26.883672 (+ 998us) server_negotiation.cc:407] Connection header received
0811 00:24:26.883811 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:26.883829 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:26.884119 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:26.884347 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:26.886164 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.887739 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.890303 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.891594 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.891752 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.892113 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:26.892581 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 00:24:26.892810 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12866,"thread_start_us":12670,"threads_started":1}
I20260811 00:24:26.894064 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.869410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:33237 (local address 127.0.0.1:41162)
0811 00:24:26.883260 (+ 13850us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:26.883295 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:26.883541 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:26.884729 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:26.884739 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:26.884806 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:26.885940 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.885956 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.887877 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.887882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.890020 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.890036 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.891844 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.891852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.892240 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.892328 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 00:24:26.893491 (+ 1163us) client_negotiation.cc:240] Negotiation successful
0811 00:24:26.893641 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13478,"mutex_wait_us":645,"spinlock_wait_cycles":23296,"thread_start_us":13280,"threads_started":1}
I20260811 00:24:26.896695 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.884044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:36963 (local address 127.0.0.1:58526)
0811 00:24:26.885982 (+ 1938us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:26.886009 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:26.886129 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:26.887109 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:26.887113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:26.887143 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:26.887562 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.887574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.889821 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.889837 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.892092 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:26.892107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.895777 (+ 3670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.895785 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:26.895990 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.896029 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:24:26.896135 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 00:24:26.896281 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1792,"mutex_wait_us":3,"thread_start_us":1591,"threads_started":1}
I20260811 00:24:26.898162 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:26.884123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58526 (local address 127.16.71.130:36963)
0811 00:24:26.885537 (+ 1414us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:26.885547 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:26.886251 (+ 704us) server_negotiation.cc:407] Connection header received
0811 00:24:26.886387 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:26.886396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:26.886678 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:26.886923 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:26.888007 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.889601 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.892359 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:26.895538 (+ 3179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:26.895705 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:26.896036 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:26.896739 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 00:24:26.897801 (+ 1062us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1269,"thread_start_us":1138,"threads_started":1}
I20260811 00:24:26.897938 18670 tablet_service.cc:1467] Processing CreateTablet for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 (DEFAULT_TABLE table=test-table [id=8762a1242b1943c59fc252876457667b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:26.897928 18229 tablet_service.cc:1467] Processing CreateTablet for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 (DEFAULT_TABLE table=test-table [id=8762a1242b1943c59fc252876457667b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:26.899585 18670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:26.900175 18229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:26.902222 18455 tablet_service.cc:1467] Processing CreateTablet for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 (DEFAULT_TABLE table=test-table [id=8762a1242b1943c59fc252876457667b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:26.903867 18455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:26.914063 18827 tablet_bootstrap.cc:492] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Bootstrap starting.
I20260811 00:24:26.917490 18829 tablet_bootstrap.cc:492] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Bootstrap starting.
I20260811 00:24:26.919586 18827 tablet_bootstrap.cc:654] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:26.920264 18830 tablet_bootstrap.cc:492] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Bootstrap starting.
I20260811 00:24:26.921381 18827 log.cc:824] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:26.921921 18829 tablet_bootstrap.cc:654] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:26.923139 18829 log.cc:824] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:26.923981 18830 tablet_bootstrap.cc:654] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:26.925235 18827 tablet_bootstrap.cc:492] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: No bootstrap required, opened a new log
I20260811 00:24:26.925426 18830 log.cc:824] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:26.925599 18827 ts_tablet_manager.cc:1397] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260811 00:24:26.927091 18829 tablet_bootstrap.cc:492] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: No bootstrap required, opened a new log
I20260811 00:24:26.927415 18829 ts_tablet_manager.cc:1397] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 00:24:26.931362 18830 tablet_bootstrap.cc:492] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: No bootstrap required, opened a new log
I20260811 00:24:26.931656 18830 ts_tablet_manager.cc:1397] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 00:24:26.938061 18829 raft_consensus.cc:348] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:26.938062 18827 raft_consensus.cc:348] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:26.938627 18829 raft_consensus.cc:374] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:26.938627 18827 raft_consensus.cc:374] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:26.938743 18829 raft_consensus.cc:729] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2304a6d192e1475ca74e4e1deddf6f08, State: Initialized, Role: FOLLOWER
I20260811 00:24:26.938743 18827 raft_consensus.cc:729] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 767bc9f5ff684cc2baf572863bd89c95, State: Initialized, Role: FOLLOWER
I20260811 00:24:26.939510 18827 consensus_queue.cc:260] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:26.939507 18829 consensus_queue.cc:260] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:26.940135 18830 raft_consensus.cc:348] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:26.940682 18830 raft_consensus.cc:374] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:26.940800 18830 raft_consensus.cc:729] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 292bee60b3bd460bb8279496a51b75ee, State: Initialized, Role: FOLLOWER
I20260811 00:24:26.941604 18794 heartbeater.cc:503] Master 127.16.71.190:32883 was elected leader, sending a full tablet report...
I20260811 00:24:26.941695 18827 ts_tablet_manager.cc:1428] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260811 00:24:26.942245 18830 consensus_queue.cc:260] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:26.943084 18829 ts_tablet_manager.cc:1428] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Time spent starting tablet: real 0.016s user 0.006s sys 0.010s
I20260811 00:24:26.944429 18557 heartbeater.cc:503] Master 127.16.71.190:32883 was elected leader, sending a full tablet report...
I20260811 00:24:26.944895 18830 ts_tablet_manager.cc:1428] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260811 00:24:27.007081 18840 raft_consensus.cc:482] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:24:27.007501 18840 raft_consensus.cc:504] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:27.009451 18840 leader_election.cc:288] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2304a6d192e1475ca74e4e1deddf6f08 (127.16.71.131:42173), 292bee60b3bd460bb8279496a51b75ee (127.16.71.130:36963)
I20260811 00:24:27.017884 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.009886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:44757 (local address 127.16.71.131:42173)
0811 00:24:27.010075 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:27.010080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:27.010914 (+ 834us) server_negotiation.cc:407] Connection header received
0811 00:24:27.011219 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:27.011226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:27.011321 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:27.011442 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:27.012221 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.013235 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.015690 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.016692 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.016851 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.016914 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:27.017434 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 00:24:27.017610 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:24:27.017931 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.010006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:42173 (local address 127.16.71.129:44757)
0811 00:24:27.010752 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:27.010774 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:27.011025 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:27.011609 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:27.011614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:27.011643 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:27.012047 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:27.012061 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.013419 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.013425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:27.015398 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:27.015422 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.016972 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.016980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:27.017189 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.017231 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 00:24:27.017373 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 00:24:27.017540 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":427,"threads_started":1}
W20260811 00:24:27.018193 18345 tablet.cc:2369] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:27.018584 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.011286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:53237 (local address 127.16.71.130:36963)
0811 00:24:27.011512 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:27.011518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:27.011732 (+ 214us) server_negotiation.cc:407] Connection header received
0811 00:24:27.011904 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:27.011909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:27.012008 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:27.012129 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:27.012900 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.013962 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.016016 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.017435 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.017618 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.017692 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:27.018153 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 00:24:27.018308 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:24:27.019821 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.010955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:36963 (local address 127.16.71.129:53237)
0811 00:24:27.011623 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:27.011639 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:27.011768 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:27.012168 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:27.012173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:27.012202 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:27.012608 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:27.012653 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.014177 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.014185 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:27.015732 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:27.015752 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.017714 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.017722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:27.017908 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.017948 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:24:27.018806 (+ 858us) client_negotiation.cc:240] Negotiation successful
0811 00:24:27.018992 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":3,"thread_start_us":321,"threads_started":1}
I20260811 00:24:27.019416 18713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a43ad2063ee4c9c8322cfbd87d89cd0" candidate_uuid: "767bc9f5ff684cc2baf572863bd89c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2304a6d192e1475ca74e4e1deddf6f08" is_pre_election: true
I20260811 00:24:27.020515 18713 raft_consensus.cc:2454] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 767bc9f5ff684cc2baf572863bd89c95 in term 0.
I20260811 00:24:27.021363 18487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a43ad2063ee4c9c8322cfbd87d89cd0" candidate_uuid: "767bc9f5ff684cc2baf572863bd89c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "292bee60b3bd460bb8279496a51b75ee" is_pre_election: true
I20260811 00:24:27.021557 18139 leader_election.cc:302] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2304a6d192e1475ca74e4e1deddf6f08, 767bc9f5ff684cc2baf572863bd89c95; no voters:
I20260811 00:24:27.021916 18487 raft_consensus.cc:2454] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 767bc9f5ff684cc2baf572863bd89c95 in term 0.
I20260811 00:24:27.022210 18840 raft_consensus.cc:2781] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:24:27.022379 18840 raft_consensus.cc:482] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:24:27.022507 18840 raft_consensus.cc:3037] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:27.025835 18840 raft_consensus.cc:504] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
I20260811 00:24:27.026840 18840 leader_election.cc:288] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [CANDIDATE]: Term 1 election: Requested vote from peers 2304a6d192e1475ca74e4e1deddf6f08 (127.16.71.131:42173), 292bee60b3bd460bb8279496a51b75ee (127.16.71.130:36963)
I20260811 00:24:27.027432 18713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a43ad2063ee4c9c8322cfbd87d89cd0" candidate_uuid: "767bc9f5ff684cc2baf572863bd89c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2304a6d192e1475ca74e4e1deddf6f08"
I20260811 00:24:27.027542 18487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a43ad2063ee4c9c8322cfbd87d89cd0" candidate_uuid: "767bc9f5ff684cc2baf572863bd89c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "292bee60b3bd460bb8279496a51b75ee"
I20260811 00:24:27.027750 18713 raft_consensus.cc:3037] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:27.027766 18487 raft_consensus.cc:3037] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:27.031569 18487 raft_consensus.cc:2454] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 767bc9f5ff684cc2baf572863bd89c95 in term 1.
I20260811 00:24:27.031536 18713 raft_consensus.cc:2454] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 767bc9f5ff684cc2baf572863bd89c95 in term 1.
I20260811 00:24:27.032167 18139 leader_election.cc:302] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 292bee60b3bd460bb8279496a51b75ee, 767bc9f5ff684cc2baf572863bd89c95; no voters:
I20260811 00:24:27.032524 18840 raft_consensus.cc:2781] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:27.033381 18840 raft_consensus.cc:686] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 1 LEADER]: Becoming Leader. State: Replica: 767bc9f5ff684cc2baf572863bd89c95, State: Running, Role: LEADER
I20260811 00:24:27.034041 18840 consensus_queue.cc:237] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } }
W20260811 00:24:27.040304 18559 tablet.cc:2369] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:27.040170 18020 catalog_manager.cc:5495] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 767bc9f5ff684cc2baf572863bd89c95 (127.16.71.129). New cstate: current_term: 1 leader_uuid: "767bc9f5ff684cc2baf572863bd89c95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "767bc9f5ff684cc2baf572863bd89c95" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 33237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 } health_report { overall_health: UNKNOWN } } }
W20260811 00:24:27.053475 18795 tablet.cc:2369] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:27.069396 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.063024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.16.71.129:33237)
0811 00:24:27.063255 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:27.063260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:27.063281 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:24:27.063513 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:27.063517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:27.063625 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:27.063773 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:24:27.064204 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.065234 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.066885 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.067352 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.067488 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.068129 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:24:27.068737 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:24:27.068920 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:27.069094 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 00:24:27.069143 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:24:27.078979 18713 raft_consensus.cc:1261] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 1 FOLLOWER]: Refusing update from remote peer 767bc9f5ff684cc2baf572863bd89c95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:27.078985 18487 raft_consensus.cc:1261] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 1 FOLLOWER]: Refusing update from remote peer 767bc9f5ff684cc2baf572863bd89c95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:27.080212 18853 consensus_queue.cc:1035] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "292bee60b3bd460bb8279496a51b75ee" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:27.080730 18840 consensus_queue.cc:1035] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2304a6d192e1475ca74e4e1deddf6f08" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:27.108336 18020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:27.108651 18020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:27.112957 18020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c833f95d6cae44ca8040170ac37be38f: Sending DeleteTablet for 3 replicas of tablet 7a43ad2063ee4c9c8322cfbd87d89cd0
I20260811 00:24:27.114372 18229 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:27 UTC) from {username='slave'} at 127.0.0.1:41162
I20260811 00:24:27.114439 18670 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:27 UTC) from {username='slave'} at 127.0.0.1:34814
I20260811 00:24:27.115459 18878 tablet_replica.cc:330] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: stopping tablet replica
I20260811 00:24:27.115561 18455 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:27 UTC) from {username='slave'} at 127.0.0.1:58526
I20260811 00:24:27.116261 18878 raft_consensus.cc:2229] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:24:27.117172 18878 raft_consensus.cc:2258] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:27.117942 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18114
I20260811 00:24:27.118213 18880 tablet_replica.cc:330] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: stopping tablet replica
I20260811 00:24:27.118759 18880 raft_consensus.cc:2229] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:27.119410 18880 raft_consensus.cc:2258] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:27.121778 18880 ts_tablet_manager.cc:1905] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:27.121781 18878 ts_tablet_manager.cc:1905] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:27.131517 18882 tablet_replica.cc:330] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: stopping tablet replica
I20260811 00:24:27.131779 18880 ts_tablet_manager.cc:1918] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:27.132035 18880 log.cc:1196] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/7a43ad2063ee4c9c8322cfbd87d89cd0
I20260811 00:24:27.132134 18882 raft_consensus.cc:2229] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:27.132774 18882 raft_consensus.cc:2258] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:27.132812 18880 ts_tablet_manager.cc:1939] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 2304a6d192e1475ca74e4e1deddf6f08: Deleting consensus metadata
I20260811 00:24:27.135003 17995 catalog_manager.cc:4841] TS 2304a6d192e1475ca74e4e1deddf6f08 (127.16.71.131:42173): tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 (table test-table [id=8762a1242b1943c59fc252876457667b]) successfully deleted
I20260811 00:24:27.135870 18882 ts_tablet_manager.cc:1905] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:27.139695 18878 ts_tablet_manager.cc:1918] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:27.139894 18878 log.cc:1196] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/7a43ad2063ee4c9c8322cfbd87d89cd0
I20260811 00:24:27.140372 18878 ts_tablet_manager.cc:1939] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 767bc9f5ff684cc2baf572863bd89c95: Deleting consensus metadata
I20260811 00:24:27.142328 17997 catalog_manager.cc:4841] TS 767bc9f5ff684cc2baf572863bd89c95 (127.16.71.129:33237): tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 (table test-table [id=8762a1242b1943c59fc252876457667b]) successfully deleted
I20260811 00:24:27.142746 18882 ts_tablet_manager.cc:1918] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:27.142932 18882 log.cc:1196] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/7a43ad2063ee4c9c8322cfbd87d89cd0
I20260811 00:24:27.143468 18882 ts_tablet_manager.cc:1939] T 7a43ad2063ee4c9c8322cfbd87d89cd0 P 292bee60b3bd460bb8279496a51b75ee: Deleting consensus metadata
I20260811 00:24:27.146201 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.119264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.16.71.129:33237)
0811 00:24:27.119467 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:27.119474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:27.137211 (+ 17737us) server_negotiation.cc:407] Connection header received
0811 00:24:27.137430 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:27.137436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:27.137561 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:27.137676 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:27.138250 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.139462 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.141584 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.142230 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.142370 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.144102 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:27.144113 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:27.144128 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:27.145149 (+ 1021us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:27.145237 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:27.145269 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:27.145405 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:27.145620 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 00:24:27.145750 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:24:27.146237 17995 catalog_manager.cc:4841] TS 292bee60b3bd460bb8279496a51b75ee (127.16.71.130:36963): tablet 7a43ad2063ee4c9c8322cfbd87d89cd0 (table test-table [id=8762a1242b1943c59fc252876457667b]) successfully deleted
I20260811 00:24:27.147253 18333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:27.257275 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18114
I20260811 00:24:27.295236 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18349
I20260811 00:24:27.303654 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.296186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.16.71.130:36963)
0811 00:24:27.296348 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:27.296353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:27.296372 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:24:27.296471 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:27.296475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:27.296577 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:27.296703 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:27.297125 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.298053 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.300006 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.300494 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.300749 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.301983 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:27.301995 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:27.302014 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:27.302750 (+ 736us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:27.302803 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:27.302836 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:27.302934 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:27.303152 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 00:24:27.303271 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:24:27.304380 18548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:27.395117 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18349
I20260811 00:24:27.417205 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18565
I20260811 00:24:27.424264 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:27.418080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.16.71.131:42173)
0811 00:24:27.418278 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:27.418283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:27.418405 (+ 122us) server_negotiation.cc:407] Connection header received
0811 00:24:27.418530 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:27.418534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:27.418648 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:27.418742 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:27.419145 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.419882 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.421355 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:27.421758 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:27.421859 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:27.422928 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:27.422936 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:27.422952 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:27.423559 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:27.423610 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:27.423632 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:27.423741 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:27.423849 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 00:24:27.423948 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:24:27.424942 18786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:27.514132 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18565
I20260811 00:24:27.541399 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 17968
I20260811 00:24:27.542744 18067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:27.659773 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 17968
2026-08-11T00:24:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3108 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:27 dist-test-slave-f7th krb5kdc[18899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:27 dist-test-slave-f7th krb5kdc[18899](info): set up 2 sockets
Aug 11 00:24:27 dist-test-slave-f7th krb5kdc[18899](info): commencing operation
krb5kdc: starting...
W20260811 00:24:28.995481 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:29 dist-test-slave-f7th krb5kdc[18899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:24:29.336848 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:33793
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:33793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:29.715814 18924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:29.716148 18924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:29.716269 18924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:29.723950 18924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:29.724089 18924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:29.724141 18924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:29.724225 18924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:29.724295 18924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:29.734895 18924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:33793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:33793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:29.736336 18924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:29.737767 18924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:29.751159 18933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:29.751184 18934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:29.751677 18936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:29.752964 18924 server_base.cc:1034] running on GCE node
I20260811 00:24:29.753918 18924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:29.755712 18924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:29.756959 18924 hybrid_clock.cc:648] HybridClock initialized: now 1786407869756903 us; error 52 us; skew 500 ppm
I20260811 00:24:29.757354 18924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:29 dist-test-slave-f7th krb5kdc[18899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407869, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:29.760699 18924 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:24:29.762506 18924 webserver.cc:460] Webserver started at http://127.16.71.190:41509/ using document root <none> and password file <none>
I20260811 00:24:29.763348 18924 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:29.763479 18924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:29.763803 18924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:29.767346 18924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "c6c38422a09f4aedbe736946e49c2c7e"
format_stamp: "Formatted at 2026-08-11 00:24:29 on dist-test-slave-f7th"
server_key: "68bda0629a141f3e56b95f197bf8039e"
server_key_iv: "db730417574e9dfb96eeda8b5fa4251e"
server_key_version: "encryptionkey@0"
I20260811 00:24:29.768182 18924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "c6c38422a09f4aedbe736946e49c2c7e"
format_stamp: "Formatted at 2026-08-11 00:24:29 on dist-test-slave-f7th"
server_key: "68bda0629a141f3e56b95f197bf8039e"
server_key_iv: "db730417574e9dfb96eeda8b5fa4251e"
server_key_version: "encryptionkey@0"
I20260811 00:24:29.774189 18924 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:24:29.778260 18948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:29.779819 18924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:24:29.780025 18924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "c6c38422a09f4aedbe736946e49c2c7e"
format_stamp: "Formatted at 2026-08-11 00:24:29 on dist-test-slave-f7th"
server_key: "68bda0629a141f3e56b95f197bf8039e"
server_key_iv: "db730417574e9dfb96eeda8b5fa4251e"
server_key_version: "encryptionkey@0"
I20260811 00:24:29.780256 18924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:29.809370 18924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:29.814334 18924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:29.814679 18924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:29.834313 18924 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:33793
I20260811 00:24:29.834339 19035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:33793 every 8 connection(s)
I20260811 00:24:29.835880 18924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:29.840183 19037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:29.841830 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 18924
I20260811 00:24:29.842039 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:29.842619 16670 external_mini_cluster.cc:1428] Setting key 42978a48b03e35147c93753351d229b4
I20260811 00:24:29.856442 19037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e: Bootstrap starting.
Aug 11 00:24:29 dist-test-slave-f7th krb5kdc[18899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:29.861186 19037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:29.862599 19037 log.cc:824] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:29.866923 19037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e: No bootstrap required, opened a new log
I20260811 00:24:29.871444 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:29.844816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55566 (local address 127.16.71.190:33793)
0811 00:24:29.845784 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:29.845799 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:29.845847 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:24:29.847669 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:29.847701 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:29.848240 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:29.848699 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:29.850864 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:29.852413 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:29.854585 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:29.855491 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:29.855686 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:29.861483 (+ 5797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:29.861495 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:29.861516 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:29.861575 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:29.865609 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:29.866847 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:29.866856 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:29.866866 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:29.867200 (+ 334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:29.867805 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:29.867810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:29.867813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:29.868562 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:29.868913 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:29.869570 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 00:24:29.870147 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":449,"threads_started":1}
I20260811 00:24:29.877307 19037 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c38422a09f4aedbe736946e49c2c7e" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33793 } }
I20260811 00:24:29.877764 19037 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:29.877884 19037 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c38422a09f4aedbe736946e49c2c7e, State: Initialized, Role: FOLLOWER
I20260811 00:24:29.878607 19037 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c38422a09f4aedbe736946e49c2c7e" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33793 } }
I20260811 00:24:29.878893 19037 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:29.879057 19037 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:29.879235 19037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:29.881740 19037 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c38422a09f4aedbe736946e49c2c7e" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33793 } }
I20260811 00:24:29.882370 19037 leader_election.cc:302] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6c38422a09f4aedbe736946e49c2c7e; no voters:
I20260811 00:24:29.883020 19037 leader_election.cc:288] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:29.883175 19045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:29.883929 19045 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [term 1 LEADER]: Becoming Leader. State: Replica: c6c38422a09f4aedbe736946e49c2c7e, State: Running, Role: LEADER
I20260811 00:24:29.884662 19045 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c38422a09f4aedbe736946e49c2c7e" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33793 } }
I20260811 00:24:29.885538 19037 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:29.888425 19047 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6c38422a09f4aedbe736946e49c2c7e. Latest consensus state: current_term: 1 leader_uuid: "c6c38422a09f4aedbe736946e49c2c7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c38422a09f4aedbe736946e49c2c7e" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33793 } } }
I20260811 00:24:29.888902 19047 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:29.888192 19046 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6c38422a09f4aedbe736946e49c2c7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c38422a09f4aedbe736946e49c2c7e" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33793 } } }
I20260811 00:24:29.889242 19046 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:29.891340 19054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:29.897585 19054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:29.908192 19054 catalog_manager.cc:1347] Generated new cluster ID: 0c950bbf024c48ec95c253532a7ea6f5
I20260811 00:24:29.908346 19054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:29.927656 19054 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:29.929821 19054 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:29.944707 19054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e: Generated new TSK 0
I20260811 00:24:29.946009 19054 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
Entry for principal kudu/127.16.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab.
I20260811 00:24:30.037216 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:33793
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:30.411516 19086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:30.411832 19086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:30.411934 19086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:30.419605 19086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:30.419742 19086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:30.419862 19086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:24:30.430948 19086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.129.keytab
--principal=kudu/127.16.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:33793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:30.432319 19086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:30.433768 19086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:30.448711 19097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:30.448676 19098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:30.448737 19102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:30.449478 19086 server_base.cc:1034] running on GCE node
I20260811 00:24:30.450205 19086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:30.451380 19086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:30.452714 19086 hybrid_clock.cc:648] HybridClock initialized: now 1786407870452594 us; error 46 us; skew 500 ppm
I20260811 00:24:30.453117 19086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:30 dist-test-slave-f7th krb5kdc[18899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:30.456827 19086 init.cc:377] Logged in from keytab as kudu/127.16.71.129@KRBTEST.COM (short username kudu)
I20260811 00:24:30.458779 19086 webserver.cc:460] Webserver started at http://127.16.71.129:40125/ using document root <none> and password file <none>
I20260811 00:24:30.459566 19086 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:30.459708 19086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:30.460067 19086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:30.463445 19086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "de7a3c6092a4405da39674639b86bf5a"
format_stamp: "Formatted at 2026-08-11 00:24:30 on dist-test-slave-f7th"
server_key: "0718a4e776697b714da4d0add520e50b"
server_key_iv: "7b646d33075a91b53bbc1950e1bd4278"
server_key_version: "encryptionkey@0"
I20260811 00:24:30.464313 19086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "de7a3c6092a4405da39674639b86bf5a"
format_stamp: "Formatted at 2026-08-11 00:24:30 on dist-test-slave-f7th"
server_key: "0718a4e776697b714da4d0add520e50b"
server_key_iv: "7b646d33075a91b53bbc1950e1bd4278"
server_key_version: "encryptionkey@0"
I20260811 00:24:30.470278 19086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 00:24:30.474376 19117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:30.475935 19086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:24:30.476133 19086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "de7a3c6092a4405da39674639b86bf5a"
format_stamp: "Formatted at 2026-08-11 00:24:30 on dist-test-slave-f7th"
server_key: "0718a4e776697b714da4d0add520e50b"
server_key_iv: "7b646d33075a91b53bbc1950e1bd4278"
server_key_version: "encryptionkey@0"
I20260811 00:24:30.476352 19086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:30.495791 19086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:30.500505 19086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:30.500851 19086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:30.501983 19086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:30.503724 19086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:30.503820 19086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:30.503958 19086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:30.504041 19086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:30.543423 19086 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:35221
I20260811 00:24:30.543529 19314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:35221 every 8 connection(s)
I20260811 00:24:30.544909 19086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:24:30.553884 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19086
I20260811 00:24:30.554073 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:24:30.554520 16670 external_mini_cluster.cc:1428] Setting key 2d328ecd5c43515b678efa87ff0acf21
Aug 11 00:24:30 dist-test-slave-f7th krb5kdc[18899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.129@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:30.567764 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:30.547823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:57889 (local address 127.16.71.190:33793)
0811 00:24:30.548311 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:30.548316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:30.549524 (+ 1208us) server_negotiation.cc:407] Connection header received
0811 00:24:30.551531 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:30.551537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:30.551631 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:30.551734 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:30.553811 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:30.554639 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:30.556472 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:30.556964 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:30.557087 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:30.561305 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:30.561311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:30.561315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:30.561353 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:30.564059 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:30.565198 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:30.565203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:30.565206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:30.565384 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:30.565936 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:30.565941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:30.565943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:30.566255 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:30.566377 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:30.567299 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 00:24:30.567494 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":243,"threads_started":1}
I20260811 00:24:30.569278 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:30.548451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:33793 (local address 127.16.71.129:57889)
0811 00:24:30.549339 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:30.549384 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:30.551201 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:30.552043 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:30.552054 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:30.552848 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:30.553550 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:30.553581 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:30.554868 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:30.554874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:30.556302 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:30.556317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:30.557229 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:30.557238 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:30.557500 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:30.558369 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:30.558402 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:30.560998 (+ 2596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:30.564256 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:30.564269 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:30.564292 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:30.565010 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:30.565505 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:30.565511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:30.565515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:30.565784 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:30.566393 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:30.566402 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:30.566891 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 00:24:30.567211 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 00:24:30.567795 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":397,"threads_started":1}
I20260811 00:24:30.571108 19315 heartbeater.cc:347] Connected to a master server at 127.16.71.190:33793
I20260811 00:24:30.571455 19315 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:30.572273 19315 heartbeater.cc:511] Master 127.16.71.190:33793 requested a full tablet report, sending...
I20260811 00:24:30.574911 18976 ts_manager.cc:194] Registered new tserver with Master: de7a3c6092a4405da39674639b86bf5a (127.16.71.129:35221)
I20260811 00:24:30.576980 18976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.129@KRBTEST.COM'} at 127.16.71.129:57889
WARNING: no policy specified for kudu/127.16.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
Entry for principal kudu/127.16.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab.
I20260811 00:24:30.633646 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:33793
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:30.999717 19327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:31.000101 19327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:31.000242 19327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:31.008086 19327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:31.008247 19327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:31.008412 19327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:24:31.019279 19327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.130.keytab
--principal=kudu/127.16.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:33793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:31.020792 19327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:31.022234 19327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:31.036474 19342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:31.036455 19341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:31.036496 19344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:31.037693 19327 server_base.cc:1034] running on GCE node
I20260811 00:24:31.038358 19327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:31.039399 19327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:31.040716 19327 hybrid_clock.cc:648] HybridClock initialized: now 1786407871040639 us; error 63 us; skew 500 ppm
I20260811 00:24:31.041088 19327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:31 dist-test-slave-f7th krb5kdc[18899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407871, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:31.044366 19327 init.cc:377] Logged in from keytab as kudu/127.16.71.130@KRBTEST.COM (short username kudu)
I20260811 00:24:31.046075 19327 webserver.cc:460] Webserver started at http://127.16.71.130:36321/ using document root <none> and password file <none>
I20260811 00:24:31.046762 19327 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:31.046890 19327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:31.047248 19327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:31.050446 19327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "08f3d3af18ad48c3965cff2015d39597"
format_stamp: "Formatted at 2026-08-11 00:24:31 on dist-test-slave-f7th"
server_key: "a3727705d4b4d9cca06023d06752baa6"
server_key_iv: "e955adffe2818080c0f5bd7b13377376"
server_key_version: "encryptionkey@0"
I20260811 00:24:31.051250 19327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "08f3d3af18ad48c3965cff2015d39597"
format_stamp: "Formatted at 2026-08-11 00:24:31 on dist-test-slave-f7th"
server_key: "a3727705d4b4d9cca06023d06752baa6"
server_key_iv: "e955adffe2818080c0f5bd7b13377376"
server_key_version: "encryptionkey@0"
I20260811 00:24:31.057047 19327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 00:24:31.060796 19353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:31.062252 19327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:24:31.062412 19327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "08f3d3af18ad48c3965cff2015d39597"
format_stamp: "Formatted at 2026-08-11 00:24:31 on dist-test-slave-f7th"
server_key: "a3727705d4b4d9cca06023d06752baa6"
server_key_iv: "e955adffe2818080c0f5bd7b13377376"
server_key_version: "encryptionkey@0"
I20260811 00:24:31.062604 19327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:31.084034 19327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:31.088862 19327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:31.089140 19327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:31.090158 19327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:31.091701 19327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:31.091805 19327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:31.091918 19327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:31.092020 19327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:31.125427 19327 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:37079
I20260811 00:24:31.125490 19545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:37079 every 8 connection(s)
I20260811 00:24:31.126847 19327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:24:31.128994 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19327
I20260811 00:24:31.129199 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:24:31.129781 16670 external_mini_cluster.cc:1428] Setting key 89585d2ffe9ef3e68a4a09fa4d78908c
Aug 11 00:24:31 dist-test-slave-f7th krb5kdc[18899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407871, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.130@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:31.149410 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.129692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:48985 (local address 127.16.71.190:33793)
0811 00:24:31.130136 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:31.130141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:31.131340 (+ 1199us) server_negotiation.cc:407] Connection header received
0811 00:24:31.133398 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:31.133404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:31.133495 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:31.133594 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:31.135266 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.135994 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.137676 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.138128 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.138247 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.142157 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:31.142164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:31.142169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:31.142214 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:31.145500 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:31.146684 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:31.146689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:31.146692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:31.146865 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:31.147387 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:31.147390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:31.147392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:31.147687 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:31.147837 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:31.148670 (+ 833us) server_negotiation.cc:299] Negotiation successful
0811 00:24:31.148949 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":215,"threads_started":1}
I20260811 00:24:31.150491 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.130328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:33793 (local address 127.16.71.130:48985)
0811 00:24:31.131191 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:31.131232 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:31.133094 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:31.133878 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:31.133887 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:31.134459 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:31.135076 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.135096 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.136163 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.136168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.137512 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.137527 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.138383 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.138391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.138637 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.139357 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:31.139386 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:31.141852 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:31.145727 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:31.145740 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:31.145764 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:31.146504 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:31.146989 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:31.146994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:31.146998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:31.147279 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:31.147852 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:31.147861 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:31.148246 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 00:24:31.148505 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 00:24:31.149165 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":314,"threads_started":1}
I20260811 00:24:31.151990 19547 heartbeater.cc:347] Connected to a master server at 127.16.71.190:33793
I20260811 00:24:31.152388 19547 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:31.153443 19547 heartbeater.cc:511] Master 127.16.71.190:33793 requested a full tablet report, sending...
I20260811 00:24:31.155448 18976 ts_manager.cc:194] Registered new tserver with Master: 08f3d3af18ad48c3965cff2015d39597 (127.16.71.130:37079)
I20260811 00:24:31.156344 18976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.130@KRBTEST.COM'} at 127.16.71.130:48985
WARNING: no policy specified for kudu/127.16.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
Entry for principal kudu/127.16.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab.
I20260811 00:24:31.207842 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:33793
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:31.575865 19555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:31.576279 19555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:31.576453 19555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 00:24:31.580878 19315 heartbeater.cc:503] Master 127.16.71.190:33793 was elected leader, sending a full tablet report...
W20260811 00:24:31.584822 19555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:31.584972 19555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:31.585129 19555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:24:31.596870 19555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.131.keytab
--principal=kudu/127.16.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:33793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:31.598371 19555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:31.599864 19555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:31.614663 19565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:31.614697 19573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:31.614614 19564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:31.622651 19555 server_base.cc:1034] running on GCE node
I20260811 00:24:31.623438 19555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:31.624564 19555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:31.625885 19555 hybrid_clock.cc:648] HybridClock initialized: now 1786407871625787 us; error 82 us; skew 500 ppm
I20260811 00:24:31.626286 19555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:31 dist-test-slave-f7th krb5kdc[18899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407871, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:31.629962 19555 init.cc:377] Logged in from keytab as kudu/127.16.71.131@KRBTEST.COM (short username kudu)
I20260811 00:24:31.631827 19555 webserver.cc:460] Webserver started at http://127.16.71.131:37015/ using document root <none> and password file <none>
I20260811 00:24:31.632659 19555 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:31.632808 19555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:31.633155 19555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:31.636348 19555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "f89fba88984942cbab3b611aef8755fb"
format_stamp: "Formatted at 2026-08-11 00:24:31 on dist-test-slave-f7th"
server_key: "3afcac36aad4021a0b88ad54fd48a82c"
server_key_iv: "974adeb4dd7a43fdb35898d47c8e5650"
server_key_version: "encryptionkey@0"
I20260811 00:24:31.637260 19555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "f89fba88984942cbab3b611aef8755fb"
format_stamp: "Formatted at 2026-08-11 00:24:31 on dist-test-slave-f7th"
server_key: "3afcac36aad4021a0b88ad54fd48a82c"
server_key_iv: "974adeb4dd7a43fdb35898d47c8e5650"
server_key_version: "encryptionkey@0"
I20260811 00:24:31.643756 19555 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:31.648357 19583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:31.650272 19555 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:24:31.650501 19555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "f89fba88984942cbab3b611aef8755fb"
format_stamp: "Formatted at 2026-08-11 00:24:31 on dist-test-slave-f7th"
server_key: "3afcac36aad4021a0b88ad54fd48a82c"
server_key_iv: "974adeb4dd7a43fdb35898d47c8e5650"
server_key_version: "encryptionkey@0"
I20260811 00:24:31.650736 19555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:31.682264 19555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:31.687475 19555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:31.687789 19555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:31.688949 19555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:31.690508 19555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:31.690594 19555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:31.690721 19555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:31.690775 19555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:31.725634 19555 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:38393
I20260811 00:24:31.725718 19778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:38393 every 8 connection(s)
I20260811 00:24:31.727367 19555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:24:31.736174 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19555
I20260811 00:24:31.736395 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:24:31.737040 16670 external_mini_cluster.cc:1428] Setting key 10d6861c80fe283021a2877ed7628206
Aug 11 00:24:31 dist-test-slave-f7th krb5kdc[18899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407871, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.131@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:24:31.760288 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.734349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:54329 (local address 127.16.71.190:33793)
0811 00:24:31.734933 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:31.734939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:31.736522 (+ 1583us) server_negotiation.cc:407] Connection header received
0811 00:24:31.739018 (+ 2496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:31.739024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:31.739142 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:31.739216 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:31.741950 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.743166 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.744977 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.745426 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.745535 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.753072 (+ 7537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:31.753078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:31.753082 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:31.753125 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:31.756872 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:31.757785 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:31.757789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:31.757791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:31.757974 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:31.758411 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:31.758415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:31.758417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:31.758719 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:31.758896 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:31.759708 (+ 812us) server_negotiation.cc:299] Negotiation successful
0811 00:24:31.759984 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":314,"threads_started":1}
I20260811 00:24:31.761281 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.735184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:33793 (local address 127.16.71.131:54329)
0811 00:24:31.736263 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:31.736324 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:31.738614 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:31.739749 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:31.739760 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:31.740724 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:31.741674 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.741711 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.743338 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.743343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.744798 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.744814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.745702 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.745707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.746016 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.750060 (+ 4044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:24:31.750091 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:31.752711 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:31.757039 (+ 4328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:31.757048 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:31.757064 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:31.757624 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:31.758077 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:24:31.758081 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:24:31.758083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:24:31.758286 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:24:31.758873 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:31.758881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:31.759276 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 00:24:31.759549 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 00:24:31.760006 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":2770,"threads_started":1}
I20260811 00:24:31.762831 19780 heartbeater.cc:347] Connected to a master server at 127.16.71.190:33793
I20260811 00:24:31.763223 19780 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:31.764024 19780 heartbeater.cc:511] Master 127.16.71.190:33793 requested a full tablet report, sending...
I20260811 00:24:31.765717 18976 ts_manager.cc:194] Registered new tserver with Master: f89fba88984942cbab3b611aef8755fb (127.16.71.131:38393)
I20260811 00:24:31.766628 18976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.71.131@KRBTEST.COM'} at 127.16.71.131:54329
I20260811 00:24:31.769136 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:24:31.787451 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.773660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.16.71.190:33793)
0811 00:24:31.773903 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:31.773909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:31.774175 (+ 266us) server_negotiation.cc:407] Connection header received
0811 00:24:31.774400 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:31.774405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:31.774485 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:31.774582 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:31.776101 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.776993 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.778338 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.778780 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.778897 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.781667 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:31.781672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:31.781676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:31.781717 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:31.784436 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:31.785431 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:31.785436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:31.785439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:31.785601 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:31.786017 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:31.786022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:31.786025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:31.786407 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:31.786655 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:31.787001 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 00:24:31.787193 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:24:31.791646 18976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:24:31.799144 18976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:24:31.837220 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.812453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:37079 (local address 127.0.0.1:54962)
0811 00:24:31.825210 (+ 12757us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:31.825284 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:31.825567 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:31.827169 (+ 1602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:31.827178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:31.827215 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:31.827704 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.827721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.830062 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.830069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.833107 (+ 3038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.833129 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.836101 (+ 2972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.836107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.836378 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.836439 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 00:24:31.836580 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 00:24:31.836816 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12430,"thread_start_us":13346,"threads_started":1}
I20260811 00:24:31.837484 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.813026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.16.71.129:35221)
0811 00:24:31.824244 (+ 11218us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:31.824257 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:31.826200 (+ 1943us) server_negotiation.cc:407] Connection header received
0811 00:24:31.826542 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:31.826551 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:31.826791 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:31.827000 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:31.828102 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.829953 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.834413 (+ 4460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.835939 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.836089 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.836315 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:31.836936 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 00:24:31.837144 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10970,"thread_start_us":11233,"threads_started":1}
I20260811 00:24:31.838369 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.812553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.16.71.130:37079)
0811 00:24:31.826170 (+ 13617us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:31.826179 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:31.826204 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:24:31.826291 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:31.826299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:31.826593 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:31.826800 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:31.827975 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.829845 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.833299 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.835594 (+ 2295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.835724 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.835948 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:31.837811 (+ 1863us) server_negotiation.cc:299] Negotiation successful
0811 00:24:31.838035 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13462,"thread_start_us":13331,"threads_started":1}
I20260811 00:24:31.838924 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.812595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35221 (local address 127.0.0.1:54772)
0811 00:24:31.826070 (+ 13475us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:31.826089 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:31.826333 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:31.827362 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:31.827368 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:31.827404 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:31.827901 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.827913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.830190 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.830196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.834175 (+ 3979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.834193 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.836474 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.836481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.836702 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.836743 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:24:31.838182 (+ 1439us) client_negotiation.cc:240] Negotiation successful
0811 00:24:31.838339 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13354,"mutex_wait_us":1583,"thread_start_us":12128,"threads_started":1}
I20260811 00:24:31.842310 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.826507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:38393 (local address 127.0.0.1:40224)
0811 00:24:31.831952 (+ 5445us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:31.831970 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:31.832129 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:31.833188 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:31.833192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:31.833222 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:31.833723 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.833739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.835653 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.835659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.837862 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:31.837881 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.841443 (+ 3562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.841452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:31.841657 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.841696 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:24:31.841797 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 00:24:31.841954 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5303,"thread_start_us":5122,"threads_started":1}
I20260811 00:24:31.843283 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:31.826662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.16.71.131:38393)
0811 00:24:31.831397 (+ 4735us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:31.831409 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:31.832092 (+ 683us) server_negotiation.cc:407] Connection header received
0811 00:24:31.832298 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:31.832306 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:31.832660 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:31.832878 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:31.833931 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.835456 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.838063 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:31.839611 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:31.839742 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:31.839972 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:31.842736 (+ 2764us) server_negotiation.cc:299] Negotiation successful
0811 00:24:31.842943 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4510,"thread_start_us":4321,"threads_started":1}
I20260811 00:24:31.843633 19196 tablet_service.cc:1467] Processing CreateTablet for tablet 784de5557b75492199dc5982a81e4f13 (DEFAULT_TABLE table=test-table [id=9151dc03a53b4e6a80b3cbf9d23661fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:31.843633 19432 tablet_service.cc:1467] Processing CreateTablet for tablet 784de5557b75492199dc5982a81e4f13 (DEFAULT_TABLE table=test-table [id=9151dc03a53b4e6a80b3cbf9d23661fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:31.845506 19432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 784de5557b75492199dc5982a81e4f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:31.845497 19196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 784de5557b75492199dc5982a81e4f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:31.848929 19661 tablet_service.cc:1467] Processing CreateTablet for tablet 784de5557b75492199dc5982a81e4f13 (DEFAULT_TABLE table=test-table [id=9151dc03a53b4e6a80b3cbf9d23661fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:31.851028 19661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 784de5557b75492199dc5982a81e4f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:31.865512 19812 tablet_bootstrap.cc:492] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Bootstrap starting.
I20260811 00:24:31.867574 19813 tablet_bootstrap.cc:492] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Bootstrap starting.
I20260811 00:24:31.871176 19812 tablet_bootstrap.cc:654] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:31.872537 19813 tablet_bootstrap.cc:654] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:31.872753 19812 log.cc:824] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:31.873656 19814 tablet_bootstrap.cc:492] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Bootstrap starting.
I20260811 00:24:31.874013 19813 log.cc:824] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:31.877013 19812 tablet_bootstrap.cc:492] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: No bootstrap required, opened a new log
I20260811 00:24:31.877278 19813 tablet_bootstrap.cc:492] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: No bootstrap required, opened a new log
I20260811 00:24:31.877416 19812 ts_tablet_manager.cc:1397] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.002s
I20260811 00:24:31.877560 19813 ts_tablet_manager.cc:1397] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 00:24:31.877592 19814 tablet_bootstrap.cc:654] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:31.878952 19814 log.cc:824] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:31.882601 19814 tablet_bootstrap.cc:492] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: No bootstrap required, opened a new log
I20260811 00:24:31.882944 19814 ts_tablet_manager.cc:1397] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 00:24:31.886644 19812 raft_consensus.cc:348] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:31.887157 19812 raft_consensus.cc:374] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:31.887256 19812 raft_consensus.cc:729] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08f3d3af18ad48c3965cff2015d39597, State: Initialized, Role: FOLLOWER
I20260811 00:24:31.887888 19812 consensus_queue.cc:260] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:31.888363 19813 raft_consensus.cc:348] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:31.888902 19813 raft_consensus.cc:374] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:31.889025 19813 raft_consensus.cc:729] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de7a3c6092a4405da39674639b86bf5a, State: Initialized, Role: FOLLOWER
I20260811 00:24:31.889158 19547 heartbeater.cc:503] Master 127.16.71.190:33793 was elected leader, sending a full tablet report...
I20260811 00:24:31.889854 19812 ts_tablet_manager.cc:1428] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260811 00:24:31.889777 19813 consensus_queue.cc:260] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:31.891697 19813 ts_tablet_manager.cc:1428] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260811 00:24:31.894130 19814 raft_consensus.cc:348] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:31.894593 19814 raft_consensus.cc:374] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:31.894712 19814 raft_consensus.cc:729] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89fba88984942cbab3b611aef8755fb, State: Initialized, Role: FOLLOWER
I20260811 00:24:31.895426 19814 consensus_queue.cc:260] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:31.897183 19814 ts_tablet_manager.cc:1428] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Time spent starting tablet: real 0.014s user 0.007s sys 0.009s
I20260811 00:24:31.897670 19780 heartbeater.cc:503] Master 127.16.71.190:33793 was elected leader, sending a full tablet report...
W20260811 00:24:31.983819 19782 tablet.cc:2369] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:32.006742 19819 raft_consensus.cc:482] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:24:32.007107 19819 raft_consensus.cc:504] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:32.008889 19819 leader_election.cc:288] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de7a3c6092a4405da39674639b86bf5a (127.16.71.129:35221), f89fba88984942cbab3b611aef8755fb (127.16.71.131:38393)
I20260811 00:24:32.018246 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.009388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:44191 (local address 127.16.71.129:35221)
0811 00:24:32.009766 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:32.009774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:32.010175 (+ 401us) server_negotiation.cc:407] Connection header received
0811 00:24:32.010390 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:32.010396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:32.010496 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:32.010606 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:32.011355 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.012477 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.015665 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.017006 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.017197 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.017280 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:32.017834 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 00:24:32.017985 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260811 00:24:32.018244 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.009501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:58843 (local address 127.16.71.131:38393)
0811 00:24:32.009940 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:32.009946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:32.010376 (+ 430us) server_negotiation.cc:407] Connection header received
0811 00:24:32.010499 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:32.010504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:32.010577 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:32.010664 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:32.011352 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.012477 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.015665 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.017006 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.017197 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.017280 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:32.017834 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 00:24:32.018020 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298}
I20260811 00:24:32.018472 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.009368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:38393 (local address 127.16.71.130:58843)
0811 00:24:32.010276 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:32.010292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:32.010397 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:32.010689 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:32.010693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:32.010712 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:32.011171 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:32.011190 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.012661 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.012666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:32.015415 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:32.015436 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.017345 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.017350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:32.017613 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.017679 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 00:24:32.017791 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 00:24:32.017947 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":198,"thread_start_us":593,"threads_started":1}
I20260811 00:24:32.018754 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.009137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:35221 (local address 127.16.71.130:44191)
0811 00:24:32.010029 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:32.010058 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:32.010235 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:32.010632 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:32.010637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:32.010666 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:32.011221 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:32.011231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.012660 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.012666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:32.015415 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:32.015436 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.017336 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.017344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:32.017614 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.017687 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 00:24:32.017791 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 00:24:32.017959 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":254,"thread_start_us":505,"threads_started":1}
I20260811 00:24:32.019687 19694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "784de5557b75492199dc5982a81e4f13" candidate_uuid: "08f3d3af18ad48c3965cff2015d39597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89fba88984942cbab3b611aef8755fb" is_pre_election: true
I20260811 00:24:32.019682 19233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "784de5557b75492199dc5982a81e4f13" candidate_uuid: "08f3d3af18ad48c3965cff2015d39597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de7a3c6092a4405da39674639b86bf5a" is_pre_election: true
I20260811 00:24:32.020195 19694 raft_consensus.cc:2454] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08f3d3af18ad48c3965cff2015d39597 in term 0.
I20260811 00:24:32.020195 19233 raft_consensus.cc:2454] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08f3d3af18ad48c3965cff2015d39597 in term 0.
I20260811 00:24:32.020902 19355 leader_election.cc:302] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08f3d3af18ad48c3965cff2015d39597, de7a3c6092a4405da39674639b86bf5a; no voters:
I20260811 00:24:32.021310 19819 raft_consensus.cc:2781] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:24:32.021481 19819 raft_consensus.cc:482] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:24:32.021586 19819 raft_consensus.cc:3037] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:32.023905 19819 raft_consensus.cc:504] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:32.024760 19819 leader_election.cc:288] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [CANDIDATE]: Term 1 election: Requested vote from peers de7a3c6092a4405da39674639b86bf5a (127.16.71.129:35221), f89fba88984942cbab3b611aef8755fb (127.16.71.131:38393)
I20260811 00:24:32.025204 19233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "784de5557b75492199dc5982a81e4f13" candidate_uuid: "08f3d3af18ad48c3965cff2015d39597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de7a3c6092a4405da39674639b86bf5a"
I20260811 00:24:32.025352 19694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "784de5557b75492199dc5982a81e4f13" candidate_uuid: "08f3d3af18ad48c3965cff2015d39597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89fba88984942cbab3b611aef8755fb"
I20260811 00:24:32.025462 19233 raft_consensus.cc:3037] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:32.025580 19694 raft_consensus.cc:3037] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:32.028911 19694 raft_consensus.cc:2454] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08f3d3af18ad48c3965cff2015d39597 in term 1.
I20260811 00:24:32.028911 19233 raft_consensus.cc:2454] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08f3d3af18ad48c3965cff2015d39597 in term 1.
I20260811 00:24:32.029479 19355 leader_election.cc:302] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08f3d3af18ad48c3965cff2015d39597, de7a3c6092a4405da39674639b86bf5a; no voters:
I20260811 00:24:32.029843 19819 raft_consensus.cc:2781] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:32.030333 19819 raft_consensus.cc:686] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 1 LEADER]: Becoming Leader. State: Replica: 08f3d3af18ad48c3965cff2015d39597, State: Running, Role: LEADER
I20260811 00:24:32.030838 19819 consensus_queue.cc:237] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } }
I20260811 00:24:32.035713 18972 catalog_manager.cc:5495] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08f3d3af18ad48c3965cff2015d39597 (127.16.71.130). New cstate: current_term: 1 leader_uuid: "08f3d3af18ad48c3965cff2015d39597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f3d3af18ad48c3965cff2015d39597" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 37079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 } health_report { overall_health: UNKNOWN } } }
W20260811 00:24:32.047785 19318 tablet.cc:2369] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:32.095176 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.089436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54966 (local address 127.16.71.130:37079)
0811 00:24:32.089650 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:32.089657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:32.089735 (+ 78us) server_negotiation.cc:407] Connection header received
0811 00:24:32.089981 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:32.089986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:32.090080 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:32.090220 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:24:32.090641 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.091402 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.092913 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.093327 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.093445 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.093996 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:24:32.094537 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:24:32.094672 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:32.094794 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 00:24:32.094897 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:24:32.104200 19233 raft_consensus.cc:1261] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 1 FOLLOWER]: Refusing update from remote peer 08f3d3af18ad48c3965cff2015d39597: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:32.104200 19694 raft_consensus.cc:1261] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 1 FOLLOWER]: Refusing update from remote peer 08f3d3af18ad48c3965cff2015d39597: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:32.105638 19828 consensus_queue.cc:1035] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de7a3c6092a4405da39674639b86bf5a" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:32.106025 19819 consensus_queue.cc:1035] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f89fba88984942cbab3b611aef8755fb" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 38393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:32.136305 18972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:32.136694 18972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:32.141317 18972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6c38422a09f4aedbe736946e49c2c7e: Sending DeleteTablet for 3 replicas of tablet 784de5557b75492199dc5982a81e4f13
I20260811 00:24:32.142830 19661 tablet_service.cc:1514] Processing DeleteTablet for tablet 784de5557b75492199dc5982a81e4f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:32 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:40224
I20260811 00:24:32.142872 19196 tablet_service.cc:1514] Processing DeleteTablet for tablet 784de5557b75492199dc5982a81e4f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:32 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:54772
I20260811 00:24:32.143671 19432 tablet_service.cc:1514] Processing DeleteTablet for tablet 784de5557b75492199dc5982a81e4f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:32 UTC) from {username='kudu', principal='kudu/127.16.71.190@KRBTEST.COM'} at 127.0.0.1:54962
I20260811 00:24:32.145545 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19086
I20260811 00:24:32.148169 19852 tablet_replica.cc:330] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: stopping tablet replica
I20260811 00:24:32.148772 19852 raft_consensus.cc:2229] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:24:32.149572 19852 raft_consensus.cc:2258] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:32.152328 19852 ts_tablet_manager.cc:1905] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:32.162688 19852 ts_tablet_manager.cc:1918] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:32.162942 19852 log.cc:1196] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/784de5557b75492199dc5982a81e4f13
I20260811 00:24:32.163578 19852 ts_tablet_manager.cc:1939] T 784de5557b75492199dc5982a81e4f13 P 08f3d3af18ad48c3965cff2015d39597: Deleting consensus metadata
I20260811 00:24:32.165964 18954 catalog_manager.cc:4841] TS 08f3d3af18ad48c3965cff2015d39597 (127.16.71.130:37079): tablet 784de5557b75492199dc5982a81e4f13 (table test-table [id=9151dc03a53b4e6a80b3cbf9d23661fd]) successfully deleted
I20260811 00:24:32.170193 19849 tablet_replica.cc:330] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: stopping tablet replica
I20260811 00:24:32.170912 19849 raft_consensus.cc:2229] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:32.171702 19849 raft_consensus.cc:2258] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:32.172272 19850 tablet_replica.cc:330] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: stopping tablet replica
I20260811 00:24:32.172757 19850 raft_consensus.cc:2229] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:32.173260 19850 raft_consensus.cc:2258] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:32.175426 19849 ts_tablet_manager.cc:1905] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:32.175462 19850 ts_tablet_manager.cc:1905] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:24:32 dist-test-slave-f7th krb5kdc[18899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.129@KRBTEST.COM
I20260811 00:24:32.182751 19850 ts_tablet_manager.cc:1918] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:32.182982 19850 log.cc:1196] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/784de5557b75492199dc5982a81e4f13
I20260811 00:24:32.183640 19850 ts_tablet_manager.cc:1939] T 784de5557b75492199dc5982a81e4f13 P f89fba88984942cbab3b611aef8755fb: Deleting consensus metadata
I20260811 00:24:32.183923 19849 ts_tablet_manager.cc:1918] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:32.184558 19849 log.cc:1196] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407527335556-16670-0/minicluster-data/ts-0/wal/wals/784de5557b75492199dc5982a81e4f13
I20260811 00:24:32.185209 19849 ts_tablet_manager.cc:1939] T 784de5557b75492199dc5982a81e4f13 P de7a3c6092a4405da39674639b86bf5a: Deleting consensus metadata
I20260811 00:24:32.185897 18954 catalog_manager.cc:4841] TS f89fba88984942cbab3b611aef8755fb (127.16.71.131:38393): tablet 784de5557b75492199dc5982a81e4f13 (table test-table [id=9151dc03a53b4e6a80b3cbf9d23661fd]) successfully deleted
I20260811 00:24:32.187204 18951 catalog_manager.cc:4841] TS de7a3c6092a4405da39674639b86bf5a (127.16.71.129:35221): tablet 784de5557b75492199dc5982a81e4f13 (table test-table [id=9151dc03a53b4e6a80b3cbf9d23661fd]) successfully deleted
I20260811 00:24:32.189507 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.147109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.16.71.129:35221)
0811 00:24:32.147310 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:32.147316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:32.169364 (+ 22048us) server_negotiation.cc:407] Connection header received
0811 00:24:32.169588 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:32.169594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:32.169702 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:32.169840 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:32.171778 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.173045 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.175098 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.175747 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.175869 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.181887 (+ 6018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:32.181906 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:32.181930 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:32.181980 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:32.186482 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:32.187289 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:32.187298 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:32.187301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:32.187475 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:32.187861 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:32.187867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:32.187870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:32.188237 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:32.188533 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:32.188928 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 00:24:32.189153 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:24:32.190249 19305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:32.309811 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19086
I20260811 00:24:32.339031 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19327
Aug 11 00:24:32 dist-test-slave-f7th krb5kdc[18899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.130@KRBTEST.COM
I20260811 00:24:32.359488 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.340006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.16.71.130:37079)
0811 00:24:32.340221 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:32.340228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:32.340257 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:24:32.340508 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:32.340514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:32.340610 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:32.340744 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:32.342101 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.343453 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.344983 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.345719 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.345881 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.351166 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:32.351184 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:32.351204 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:32.351258 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:32.355677 (+ 4419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:32.356815 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:32.356828 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:32.356832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:32.357035 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:32.357589 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:32.357595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:32.357599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:32.358203 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:32.358573 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:32.358813 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 00:24:32.359078 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:24:32.360283 19536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:32.479735 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19327
I20260811 00:24:32.504477 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19555
Aug 11 00:24:32 dist-test-slave-f7th krb5kdc[18899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.131@KRBTEST.COM
I20260811 00:24:32.522398 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:32.505596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.16.71.131:38393)
0811 00:24:32.505822 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:32.505828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:32.505852 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:24:32.505933 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:32.505938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:32.506007 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:32.506098 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:32.507670 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.508447 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.510044 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:32.510459 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:32.510585 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:32.515728 (+ 5143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:32.515747 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:32.515768 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:32.515840 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:32.519153 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:32.520062 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:32.520071 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:32.520074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:32.520256 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:32.520695 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:32.520700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:32.520702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:32.521127 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:32.521442 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:32.521795 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 00:24:32.522035 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:24:32.523092 19771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:32.616919 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19555
I20260811 00:24:32.651757 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 18924
I20260811 00:24:32.653033 19029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:32.767359 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 18924
2026-08-11T00:24:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5105 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T00:24:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:32Z Disabled control of system clock
I20260811 00:24:32.877489 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:38989
--webserver_interface=127.16.71.190
--webserver_port=0
--builtin_ntp_servers=127.16.71.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:38989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:24:33.259553 19870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:33.259928 19870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:33.260066 19870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:33.267963 19870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:33.268112 19870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:33.268157 19870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:33.268246 19870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:33.268306 19870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:33.279119 19870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:38989
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:38989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:33.280700 19870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:33.282752 19870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:33.295612 19886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:33.295691 19890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:33.295727 19881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:33.297112 19870 server_base.cc:1034] running on GCE node
I20260811 00:24:33.298028 19870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:33.299633 19870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:33.300843 19870 hybrid_clock.cc:648] HybridClock initialized: now 1786407873300789 us; error 56 us; skew 500 ppm
I20260811 00:24:33.301244 19870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:33.303932 19870 webserver.cc:460] Webserver started at http://127.16.71.190:42651/ using document root <none> and password file <none>
I20260811 00:24:33.304791 19870 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:33.304906 19870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:33.305256 19870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:33.308511 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "e96e91c92a7b47fcba804670551bc14d"
format_stamp: "Formatted at 2026-08-11 00:24:33 on dist-test-slave-f7th"
server_key: "1a72011dd7046fa774384f25602b3fa6"
server_key_iv: "82073dfafea79567d5c83ee791a173f9"
server_key_version: "encryptionkey@0"
I20260811 00:24:33.309486 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "e96e91c92a7b47fcba804670551bc14d"
format_stamp: "Formatted at 2026-08-11 00:24:33 on dist-test-slave-f7th"
server_key: "1a72011dd7046fa774384f25602b3fa6"
server_key_iv: "82073dfafea79567d5c83ee791a173f9"
server_key_version: "encryptionkey@0"
I20260811 00:24:33.315637 19870 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:24:33.319849 19901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:33.321542 19870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:24:33.321771 19870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "e96e91c92a7b47fcba804670551bc14d"
format_stamp: "Formatted at 2026-08-11 00:24:33 on dist-test-slave-f7th"
server_key: "1a72011dd7046fa774384f25602b3fa6"
server_key_iv: "82073dfafea79567d5c83ee791a173f9"
server_key_version: "encryptionkey@0"
I20260811 00:24:33.322069 19870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:33.341030 19870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:33.342006 19870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:33.342310 19870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:33.362116 19870 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:38989
I20260811 00:24:33.362128 19991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:38989 every 8 connection(s)
I20260811 00:24:33.363544 19870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:33.367944 19996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:33.372418 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 19870
I20260811 00:24:33.372663 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:33.373116 16670 external_mini_cluster.cc:1428] Setting key 30582b37fd2e458d5e12650f4a01158c
I20260811 00:24:33.380241 19996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d: Bootstrap starting.
I20260811 00:24:33.384281 19996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:33.385776 19996 log.cc:824] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:33.389053 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:33.375201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.16.71.190:38989)
0811 00:24:33.376265 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:33.376277 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:33.376323 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:24:33.378513 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:33.378545 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:33.379115 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:33.379503 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:33.380286 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:33.381774 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:33.383783 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:33.384656 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:33.384851 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:33.386151 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:33.386159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:33.386176 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:33.387029 (+ 853us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:33.387155 (+ 126us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:33.387191 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:33.387323 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:33.387591 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 00:24:33.387942 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":373,"threads_started":1}
I20260811 00:24:33.389532 19996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d: No bootstrap required, opened a new log
I20260811 00:24:33.398456 19996 raft_consensus.cc:348] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96e91c92a7b47fcba804670551bc14d" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38989 } }
I20260811 00:24:33.398984 19996 raft_consensus.cc:374] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:33.399098 19996 raft_consensus.cc:729] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96e91c92a7b47fcba804670551bc14d, State: Initialized, Role: FOLLOWER
I20260811 00:24:33.399807 19996 consensus_queue.cc:260] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96e91c92a7b47fcba804670551bc14d" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38989 } }
I20260811 00:24:33.400104 19996 raft_consensus.cc:388] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:33.400305 19996 raft_consensus.cc:482] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:33.400799 19996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:33.403571 19996 raft_consensus.cc:504] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96e91c92a7b47fcba804670551bc14d" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38989 } }
I20260811 00:24:33.404158 19996 leader_election.cc:302] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e96e91c92a7b47fcba804670551bc14d; no voters:
I20260811 00:24:33.404896 19996 leader_election.cc:288] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:33.405099 20002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:33.405893 20002 raft_consensus.cc:686] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [term 1 LEADER]: Becoming Leader. State: Replica: e96e91c92a7b47fcba804670551bc14d, State: Running, Role: LEADER
I20260811 00:24:33.406497 20002 consensus_queue.cc:237] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96e91c92a7b47fcba804670551bc14d" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38989 } }
I20260811 00:24:33.407320 19996 sys_catalog.cc:564] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:33.409612 20003 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e96e91c92a7b47fcba804670551bc14d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96e91c92a7b47fcba804670551bc14d" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38989 } } }
I20260811 00:24:33.409637 20004 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e96e91c92a7b47fcba804670551bc14d. Latest consensus state: current_term: 1 leader_uuid: "e96e91c92a7b47fcba804670551bc14d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96e91c92a7b47fcba804670551bc14d" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 38989 } } }
I20260811 00:24:33.410387 20004 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:33.410238 20003 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:33.412094 20012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:33.418282 20012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:33.428416 20012 catalog_manager.cc:1347] Generated new cluster ID: 432618df352c4c558829a148468f3b84
I20260811 00:24:33.428584 20012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:33.450314 20012 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:33.452666 20012 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:33.468561 20012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d: Generated new TSK 0
I20260811 00:24:33.469621 20012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:33.480450 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.129:0
--local_ip_for_outbound_sockets=127.16.71.129
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38989
--builtin_ntp_servers=127.16.71.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:24:33.902626 20039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:33.902966 20039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:33.903060 20039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:33.910802 20039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:33.910959 20039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:33.911106 20039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.129
I20260811 00:24:33.921056 20039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.71.129
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.129
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:33.922465 20039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:33.924403 20039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:33.939007 20055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:33.939004 20045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:33.939343 20049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:33.940373 20039 server_base.cc:1034] running on GCE node
I20260811 00:24:33.941156 20039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:33.942287 20039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:33.943547 20039 hybrid_clock.cc:648] HybridClock initialized: now 1786407873943482 us; error 47 us; skew 500 ppm
I20260811 00:24:33.944018 20039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:33.947098 20039 webserver.cc:460] Webserver started at http://127.16.71.129:38789/ using document root <none> and password file <none>
I20260811 00:24:33.947943 20039 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:33.948052 20039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:33.948392 20039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:33.951884 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data/instance:
uuid: "f4c109f5550c48838a09dddaf5b6f2dc"
format_stamp: "Formatted at 2026-08-11 00:24:33 on dist-test-slave-f7th"
server_key: "7e22f4ba6b9f99ff8730f6a9d183ac37"
server_key_iv: "393c2dab32e74706a76d6e4fcbd368cd"
server_key_version: "encryptionkey@0"
I20260811 00:24:33.952915 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance:
uuid: "f4c109f5550c48838a09dddaf5b6f2dc"
format_stamp: "Formatted at 2026-08-11 00:24:33 on dist-test-slave-f7th"
server_key: "7e22f4ba6b9f99ff8730f6a9d183ac37"
server_key_iv: "393c2dab32e74706a76d6e4fcbd368cd"
server_key_version: "encryptionkey@0"
I20260811 00:24:33.959085 20039 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:33.963445 20069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:33.965108 20039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:24:33.965307 20039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/wal
uuid: "f4c109f5550c48838a09dddaf5b6f2dc"
format_stamp: "Formatted at 2026-08-11 00:24:33 on dist-test-slave-f7th"
server_key: "7e22f4ba6b9f99ff8730f6a9d183ac37"
server_key_iv: "393c2dab32e74706a76d6e4fcbd368cd"
server_key_version: "encryptionkey@0"
I20260811 00:24:33.965551 20039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:33.989249 20039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:33.990182 20039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:33.990501 20039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:33.991616 20039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:33.993417 20039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:33.993522 20039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:33.993639 20039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:33.993712 20039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:34.028921 20039 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.129:41425
I20260811 00:24:34.029031 20275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.129:41425 every 8 connection(s)
I20260811 00:24:34.030337 20039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/data/info.pb
I20260811 00:24:34.036368 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 20039
I20260811 00:24:34.036556 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-0/wal/instance
I20260811 00:24:34.037032 16670 external_mini_cluster.cc:1428] Setting key 5408de9041b5b3d5ad1adc83fba9861d
I20260811 00:24:34.040159 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.130:0
--local_ip_for_outbound_sockets=127.16.71.130
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38989
--builtin_ntp_servers=127.16.71.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:24:34.044720 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:34.033353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:51049 (local address 127.16.71.190:38989)
0811 00:24:34.033878 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:34.033883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:34.035231 (+ 1348us) server_negotiation.cc:407] Connection header received
0811 00:24:34.037310 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:34.037316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:34.037414 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:34.037516 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:34.039080 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.039808 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.041625 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.042049 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.042170 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:34.043739 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:34.043745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:34.043748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:34.043769 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:34.043813 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:34.043825 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:34.043907 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:34.044197 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 00:24:34.044315 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":287,"threads_started":1}
I20260811 00:24:34.045830 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:34.033893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:38989 (local address 127.16.71.129:51049)
0811 00:24:34.035020 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:34.035074 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:34.036974 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:34.037804 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:34.037814 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:34.037923 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:34.038884 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:34.038908 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.039985 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.039989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:34.041448 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:34.041463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.042300 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.042308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:34.042561 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:34.043345 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:34.043348 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:34.043394 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:34.043496 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:34.043927 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:34.043940 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:34.043979 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:24:34.044164 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 00:24:34.044663 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":356,"threads_started":1}
I20260811 00:24:34.047720 20278 heartbeater.cc:347] Connected to a master server at 127.16.71.190:38989
I20260811 00:24:34.048070 20278 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:34.049037 20278 heartbeater.cc:511] Master 127.16.71.190:38989 requested a full tablet report, sending...
I20260811 00:24:34.051683 19934 ts_manager.cc:194] Registered new tserver with Master: f4c109f5550c48838a09dddaf5b6f2dc (127.16.71.129:41425)
I20260811 00:24:34.053690 19934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.129:51049
W20260811 00:24:34.406057 20290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:34.406384 20290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:34.406509 20290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:34.414320 20290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:34.414459 20290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:34.414619 20290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.130
I20260811 00:24:34.424230 20290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.71.130
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.130
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:34.425697 20290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:34.427835 20290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:34.442497 20299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:34.442557 20309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:34.442597 20301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:34.446586 20290 server_base.cc:1034] running on GCE node
I20260811 00:24:34.447371 20290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:34.448472 20290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:34.449738 20290 hybrid_clock.cc:648] HybridClock initialized: now 1786407874449688 us; error 37 us; skew 500 ppm
I20260811 00:24:34.450163 20290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:34.452875 20290 webserver.cc:460] Webserver started at http://127.16.71.130:37551/ using document root <none> and password file <none>
I20260811 00:24:34.453650 20290 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:34.453753 20290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:34.454078 20290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:34.457489 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data/instance:
uuid: "6cea23ae3a864926a81ecb818be53a01"
format_stamp: "Formatted at 2026-08-11 00:24:34 on dist-test-slave-f7th"
server_key: "ff7d7fd5791a6755cf7d4fcfc6ef6794"
server_key_iv: "b345239327b37d11a7f8407f9f24a956"
server_key_version: "encryptionkey@0"
I20260811 00:24:34.458400 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance:
uuid: "6cea23ae3a864926a81ecb818be53a01"
format_stamp: "Formatted at 2026-08-11 00:24:34 on dist-test-slave-f7th"
server_key: "ff7d7fd5791a6755cf7d4fcfc6ef6794"
server_key_iv: "b345239327b37d11a7f8407f9f24a956"
server_key_version: "encryptionkey@0"
I20260811 00:24:34.464920 20290 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 00:24:34.469298 20317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:34.470995 20290 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:24:34.471191 20290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal
uuid: "6cea23ae3a864926a81ecb818be53a01"
format_stamp: "Formatted at 2026-08-11 00:24:34 on dist-test-slave-f7th"
server_key: "ff7d7fd5791a6755cf7d4fcfc6ef6794"
server_key_iv: "b345239327b37d11a7f8407f9f24a956"
server_key_version: "encryptionkey@0"
I20260811 00:24:34.471414 20290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:34.492141 20290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:34.493114 20290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:34.493417 20290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:34.494488 20290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:34.496142 20290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:34.496236 20290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:34.496346 20290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:34.496413 20290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:34.531131 20290 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.130:45659
I20260811 00:24:34.531184 20513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.130:45659 every 8 connection(s)
I20260811 00:24:34.532524 20290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/data/info.pb
I20260811 00:24:34.536422 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 20290
I20260811 00:24:34.536593 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal/instance
I20260811 00:24:34.537081 16670 external_mini_cluster.cc:1428] Setting key d55755ff53304d7fe55765e5ecc54dbe
I20260811 00:24:34.540283 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.71.131:0
--local_ip_for_outbound_sockets=127.16.71.131
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38989
--builtin_ntp_servers=127.16.71.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:24:34.546530 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:34.535319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.130:48989 (local address 127.16.71.190:38989)
0811 00:24:34.535505 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:34.535510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:34.537206 (+ 1696us) server_negotiation.cc:407] Connection header received
0811 00:24:34.539062 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:34.539066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:34.539135 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:34.539214 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:34.540441 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.541497 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.543286 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.543896 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.544035 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:34.545614 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:34.545621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:34.545625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:34.545651 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:34.545699 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:34.545715 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:34.545806 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:34.546050 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 00:24:34.546168 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:24:34.547761 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:34.535838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:38989 (local address 127.16.71.130:48989)
0811 00:24:34.536990 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:34.537037 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:34.538781 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:34.539458 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:34.539468 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:34.539583 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:34.540234 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:34.540254 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.541706 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.541711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:34.543093 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:34.543109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:34.544156 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:34.544161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:34.544440 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:34.545224 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:34.545227 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:34.545275 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:34.545377 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:34.545818 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:34.545829 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:34.545836 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:24:34.546017 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 00:24:34.546425 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":360,"threads_started":1}
I20260811 00:24:34.549567 20515 heartbeater.cc:347] Connected to a master server at 127.16.71.190:38989
I20260811 00:24:34.549924 20515 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:34.550803 20515 heartbeater.cc:511] Master 127.16.71.190:38989 requested a full tablet report, sending...
I20260811 00:24:34.552525 19934 ts_manager.cc:194] Registered new tserver with Master: 6cea23ae3a864926a81ecb818be53a01 (127.16.71.130:45659)
I20260811 00:24:34.553431 19934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.130:48989
W20260811 00:24:34.909521 20521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:34.909839 20521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:34.909957 20521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:34.918033 20521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:34.918180 20521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:34.918330 20521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.71.131
I20260811 00:24:34.928341 20521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.71.131
--webserver_port=0
--tserver_master_addrs=127.16.71.190:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.71.131
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:34.929841 20521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:34.932193 20521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:34.946568 20531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:34.947448 20532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:34.947854 20535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:34.949041 20521 server_base.cc:1034] running on GCE node
I20260811 00:24:34.949833 20521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:34.951006 20521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:34.952327 20521 hybrid_clock.cc:648] HybridClock initialized: now 1786407874952253 us; error 63 us; skew 500 ppm
I20260811 00:24:34.952826 20521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:24:34.956749 20521 webserver.cc:460] Webserver started at http://127.16.71.131:46155/ using document root <none> and password file <none>
I20260811 00:24:34.957930 20521 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:34.958104 20521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:34.958622 20521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:34.962795 20521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data/instance:
uuid: "44c4f8ca786d4b91b78f8803290c78cf"
format_stamp: "Formatted at 2026-08-11 00:24:34 on dist-test-slave-f7th"
server_key: "165f0949984eed7db2182d265cad3ed7"
server_key_iv: "08a41c4db23fdea4768679653737efb3"
server_key_version: "encryptionkey@0"
I20260811 00:24:34.963950 20521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance:
uuid: "44c4f8ca786d4b91b78f8803290c78cf"
format_stamp: "Formatted at 2026-08-11 00:24:34 on dist-test-slave-f7th"
server_key: "165f0949984eed7db2182d265cad3ed7"
server_key_iv: "08a41c4db23fdea4768679653737efb3"
server_key_version: "encryptionkey@0"
I20260811 00:24:34.972409 20521 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260811 00:24:34.977193 20547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:34.978952 20521 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:24:34.979167 20521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal
uuid: "44c4f8ca786d4b91b78f8803290c78cf"
format_stamp: "Formatted at 2026-08-11 00:24:34 on dist-test-slave-f7th"
server_key: "165f0949984eed7db2182d265cad3ed7"
server_key_iv: "08a41c4db23fdea4768679653737efb3"
server_key_version: "encryptionkey@0"
I20260811 00:24:34.979429 20521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:34.993144 20521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:34.994060 20521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:34.994370 20521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:34.995404 20521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:24:34.997036 20521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:24:34.997146 20521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:34.997267 20521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:24:34.997339 20521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:35.030551 20521 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.131:45661
I20260811 00:24:35.030587 20730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.131:45661 every 8 connection(s)
I20260811 00:24:35.031961 20521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/data/info.pb
I20260811 00:24:35.037051 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 20521
I20260811 00:24:35.037252 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal/instance
I20260811 00:24:35.037808 16670 external_mini_cluster.cc:1428] Setting key 3c752363b264c7579832070c768714fd
I20260811 00:24:35.045758 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.034700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.131:56707 (local address 127.16.71.190:38989)
0811 00:24:35.034910 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.034918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.036422 (+ 1504us) server_negotiation.cc:407] Connection header received
0811 00:24:35.038768 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.038774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.038880 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.039031 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:35.040245 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.041119 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.042768 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.043183 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.043301 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.044833 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:35.044838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:35.044841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:35.044862 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:35.044898 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:35.044910 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:35.045040 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.045280 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.045393 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:24:35.046864 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.035242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:38989 (local address 127.16.71.131:56707)
0811 00:24:35.036240 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.036288 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:35.038488 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:35.039314 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:35.039324 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:35.039438 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:24:35.040065 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.040084 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.041273 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.041277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.042603 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.042618 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.043436 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.043446 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.043690 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.044445 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:24:35.044448 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:24:35.044496 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:24:35.044591 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:24:35.045055 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:24:35.045068 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:24:35.045077 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:24:35.045251 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 00:24:35.045708 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":333,"threads_started":1}
I20260811 00:24:35.048382 20731 heartbeater.cc:347] Connected to a master server at 127.16.71.190:38989
I20260811 00:24:35.048740 20731 heartbeater.cc:464] Registering TS with master...
I20260811 00:24:35.049531 20731 heartbeater.cc:511] Master 127.16.71.190:38989 requested a full tablet report, sending...
I20260811 00:24:35.051275 19934 ts_manager.cc:194] Registered new tserver with Master: 44c4f8ca786d4b91b78f8803290c78cf (127.16.71.131:45661)
I20260811 00:24:35.052244 19934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.71.131:56707
I20260811 00:24:35.053643 16670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:24:35.057974 20278 heartbeater.cc:503] Master 127.16.71.190:38989 was elected leader, sending a full tablet report...
I20260811 00:24:35.064599 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.058168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.16.71.190:38989)
0811 00:24:35.058440 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.058446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.059003 (+ 557us) server_negotiation.cc:407] Connection header received
0811 00:24:35.059181 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.059185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.059251 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.059338 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:35.059880 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.060606 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.062113 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.062548 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.062657 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.063863 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:35.063870 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:35.063874 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:35.063902 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:35.063942 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:35.063957 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:35.064042 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.064207 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.064298 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:24:35.069154 19934 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:24:35.076387 19934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:24:35.112500 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.090024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:45661 (local address 127.0.0.1:52516)
0811 00:24:35.097350 (+ 7326us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.097379 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:35.097547 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:35.104826 (+ 7279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:35.104832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:35.104866 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:35.105287 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.105298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.107294 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.107300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.109418 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.109434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.111187 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.111194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.111464 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.111539 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 00:24:35.111660 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 00:24:35.111945 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":141,"thread_start_us":851,"threads_started":1}
I20260811 00:24:35.112569 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.091321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44172 (local address 127.16.71.130:45659)
0811 00:24:35.102727 (+ 11406us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.102738 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.103791 (+ 1053us) server_negotiation.cc:407] Connection header received
0811 00:24:35.103911 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.103919 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.104146 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.104342 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:35.105532 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.106933 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.109961 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.111398 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.111549 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.111814 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.112020 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.112206 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11168,"thread_start_us":11429,"threads_started":1}
I20260811 00:24:35.112502 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.091128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:45659 (local address 127.0.0.1:44172)
0811 00:24:35.103476 (+ 12348us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.103498 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:35.103648 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:35.104661 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:35.104668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:35.104703 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:35.105269 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.105287 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.107097 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.107101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.109714 (+ 2613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.109730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.111598 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.111604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.111814 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.111845 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:24:35.111966 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 00:24:35.112090 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12207,"mutex_wait_us":127,"thread_start_us":12046,"threads_started":1}
I20260811 00:24:35.113684 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.090305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57886)
0811 00:24:35.096394 (+ 6089us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.096464 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:35.096759 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:35.105310 (+ 8551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:35.105315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:35.105340 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:35.105744 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.105757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.107786 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.107792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.110050 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.110064 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.112897 (+ 2833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.112905 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.113065 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.113095 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:24:35.113190 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 00:24:35.113329 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5706,"mutex_wait_us":4,"thread_start_us":848,"threads_started":1}
I20260811 00:24:35.113775 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.090997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.16.71.129:41425)
0811 00:24:35.104499 (+ 13502us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.104507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.104536 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:24:35.104659 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.104666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.104865 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.105077 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:35.105999 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.107622 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.110276 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.112261 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.112420 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.112755 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.113254 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.113426 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13311,"thread_start_us":13175,"threads_started":1}
I20260811 00:24:35.114005 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.090558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.16.71.131:45661)
0811 00:24:35.103330 (+ 12772us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.103338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.103395 (+ 57us) server_negotiation.cc:407] Connection header received
0811 00:24:35.103523 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.103532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.103841 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.104824 (+ 983us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:35.105510 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.107071 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.109661 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.110953 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.113087 (+ 2134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.113382 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.113458 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.113637 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12591,"thread_start_us":13501,"threads_started":1}
I20260811 00:24:35.117712 20626 tablet_service.cc:1467] Processing CreateTablet for tablet 4adc47142f3845698e531258d598f9ef (DEFAULT_TABLE table=test-table [id=dadb6a2894c34cb69a0b52a2820d52d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:35.117530 20397 tablet_service.cc:1467] Processing CreateTablet for tablet 4adc47142f3845698e531258d598f9ef (DEFAULT_TABLE table=test-table [id=dadb6a2894c34cb69a0b52a2820d52d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:35.117530 20160 tablet_service.cc:1467] Processing CreateTablet for tablet 4adc47142f3845698e531258d598f9ef (DEFAULT_TABLE table=test-table [id=dadb6a2894c34cb69a0b52a2820d52d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:24:35.119155 20626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4adc47142f3845698e531258d598f9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:35.119217 20160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4adc47142f3845698e531258d598f9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:35.119216 20397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4adc47142f3845698e531258d598f9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:35.132217 20773 tablet_bootstrap.cc:492] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Bootstrap starting.
I20260811 00:24:35.134591 20775 tablet_bootstrap.cc:492] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: Bootstrap starting.
I20260811 00:24:35.136415 20773 tablet_bootstrap.cc:654] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:35.137980 20773 log.cc:824] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:35.138716 20775 tablet_bootstrap.cc:654] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:35.140208 20778 tablet_bootstrap.cc:492] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Bootstrap starting.
I20260811 00:24:35.140242 20775 log.cc:824] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:35.144111 20773 tablet_bootstrap.cc:492] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: No bootstrap required, opened a new log
I20260811 00:24:35.144464 20775 tablet_bootstrap.cc:492] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: No bootstrap required, opened a new log
I20260811 00:24:35.144467 20773 ts_tablet_manager.cc:1397] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260811 00:24:35.144807 20775 ts_tablet_manager.cc:1397] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 00:24:35.145308 20778 tablet_bootstrap.cc:654] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:35.146440 20778 log.cc:824] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:35.155555 20778 tablet_bootstrap.cc:492] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: No bootstrap required, opened a new log
I20260811 00:24:35.155905 20778 ts_tablet_manager.cc:1397] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260811 00:24:35.155802 20775 raft_consensus.cc:348] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.155965 20773 raft_consensus.cc:348] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.156355 20775 raft_consensus.cc:374] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:35.156386 20773 raft_consensus.cc:374] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:35.156476 20773 raft_consensus.cc:729] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c4f8ca786d4b91b78f8803290c78cf, State: Initialized, Role: FOLLOWER
I20260811 00:24:35.156476 20775 raft_consensus.cc:729] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c109f5550c48838a09dddaf5b6f2dc, State: Initialized, Role: FOLLOWER
I20260811 00:24:35.157210 20775 consensus_queue.cc:260] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.157202 20773 consensus_queue.cc:260] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.159192 20775 ts_tablet_manager.cc:1428] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 00:24:35.159864 20731 heartbeater.cc:503] Master 127.16.71.190:38989 was elected leader, sending a full tablet report...
I20260811 00:24:35.162101 20773 ts_tablet_manager.cc:1428] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
I20260811 00:24:35.165083 20778 raft_consensus.cc:348] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.165557 20778 raft_consensus.cc:374] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:35.165649 20778 raft_consensus.cc:729] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cea23ae3a864926a81ecb818be53a01, State: Initialized, Role: FOLLOWER
I20260811 00:24:35.166273 20778 consensus_queue.cc:260] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.167369 20515 heartbeater.cc:503] Master 127.16.71.190:38989 was elected leader, sending a full tablet report...
I20260811 00:24:35.167958 20778 ts_tablet_manager.cc:1428] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Time spent starting tablet: real 0.012s user 0.009s sys 0.005s
W20260811 00:24:35.283133 20282 tablet.cc:2369] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:24:35.283764 20734 tablet.cc:2369] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:24:35.284752 20516 tablet.cc:2369] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:24:35.299567 20787 raft_consensus.cc:482] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:24:35.299947 20787 raft_consensus.cc:504] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.302126 20787 leader_election.cc:288] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44c4f8ca786d4b91b78f8803290c78cf (127.16.71.131:45661), 6cea23ae3a864926a81ecb818be53a01 (127.16.71.130:45659)
I20260811 00:24:35.308900 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.302596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:58441 (local address 127.16.71.130:45659)
0811 00:24:35.302779 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.302784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.303471 (+ 687us) server_negotiation.cc:407] Connection header received
0811 00:24:35.303671 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.303677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.303761 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.303839 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:35.304376 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.305154 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.307174 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.307966 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.308115 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.308188 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.308462 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.308594 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:24:35.309074 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.302589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.130:45659 (local address 127.16.71.129:58441)
0811 00:24:35.303345 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.303363 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:35.303536 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:35.303877 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:35.303882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:35.303931 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:35.304238 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.304247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.305281 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.305285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.306980 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.306994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.308144 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.308150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.308295 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.308320 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 00:24:35.308413 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:24:35.308561 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":79,"spinlock_wait_cycles":3456,"thread_start_us":858,"threads_started":1}
I20260811 00:24:35.309257 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.302507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.71.129:45725 (local address 127.16.71.131:45661)
0811 00:24:35.302717 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.302724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.303883 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 00:24:35.304055 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.304059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.304120 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.304197 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:24:35.304751 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.305457 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.307403 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.308290 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.308464 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.308540 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.308881 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.309018 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:24:35.309710 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.302613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.131:45661 (local address 127.16.71.129:45725)
0811 00:24:35.303768 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.303783 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:24:35.303911 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:24:35.304247 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:24:35.304252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:24:35.304276 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:24:35.304584 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.304593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.305604 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.305609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.307209 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:24:35.307223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.308489 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.308496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:24:35.308692 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.308717 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 00:24:35.308876 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 00:24:35.309369 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":43,"spinlock_wait_cycles":24832,"thread_start_us":396,"threads_started":1}
I20260811 00:24:35.310395 20431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adc47142f3845698e531258d598f9ef" candidate_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cea23ae3a864926a81ecb818be53a01" is_pre_election: true
I20260811 00:24:35.310596 20660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adc47142f3845698e531258d598f9ef" candidate_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c4f8ca786d4b91b78f8803290c78cf" is_pre_election: true
I20260811 00:24:35.311094 20431 raft_consensus.cc:2454] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c109f5550c48838a09dddaf5b6f2dc in term 0.
I20260811 00:24:35.311124 20660 raft_consensus.cc:2454] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c109f5550c48838a09dddaf5b6f2dc in term 0.
I20260811 00:24:35.311857 20073 leader_election.cc:302] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c4f8ca786d4b91b78f8803290c78cf, f4c109f5550c48838a09dddaf5b6f2dc; no voters:
I20260811 00:24:35.312314 20787 raft_consensus.cc:2781] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:24:35.312544 20787 raft_consensus.cc:482] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:24:35.312680 20787 raft_consensus.cc:3037] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:35.314924 20787 raft_consensus.cc:504] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.315632 20787 leader_election.cc:288] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [CANDIDATE]: Term 1 election: Requested vote from peers 44c4f8ca786d4b91b78f8803290c78cf (127.16.71.131:45661), 6cea23ae3a864926a81ecb818be53a01 (127.16.71.130:45659)
I20260811 00:24:35.316253 20660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adc47142f3845698e531258d598f9ef" candidate_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c4f8ca786d4b91b78f8803290c78cf"
I20260811 00:24:35.316272 20431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adc47142f3845698e531258d598f9ef" candidate_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cea23ae3a864926a81ecb818be53a01"
I20260811 00:24:35.316532 20660 raft_consensus.cc:3037] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:35.316540 20431 raft_consensus.cc:3037] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:35.319907 20431 raft_consensus.cc:2454] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c109f5550c48838a09dddaf5b6f2dc in term 1.
I20260811 00:24:35.319907 20660 raft_consensus.cc:2454] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c109f5550c48838a09dddaf5b6f2dc in term 1.
I20260811 00:24:35.320478 20073 leader_election.cc:302] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 44c4f8ca786d4b91b78f8803290c78cf, 6cea23ae3a864926a81ecb818be53a01, f4c109f5550c48838a09dddaf5b6f2dc; no voters:
I20260811 00:24:35.320888 20787 raft_consensus.cc:2781] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:35.321374 20787 raft_consensus.cc:686] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 1 LEADER]: Becoming Leader. State: Replica: f4c109f5550c48838a09dddaf5b6f2dc, State: Running, Role: LEADER
I20260811 00:24:35.321909 20787 consensus_queue.cc:237] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } }
I20260811 00:24:35.327297 19926 catalog_manager.cc:5495] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc reported cstate change: term changed from 0 to 1, leader changed from <none> to f4c109f5550c48838a09dddaf5b6f2dc (127.16.71.129). New cstate: current_term: 1 leader_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4c109f5550c48838a09dddaf5b6f2dc" member_type: VOTER last_known_addr { host: "127.16.71.129" port: 41425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 } health_report { overall_health: UNKNOWN } } }
I20260811 00:24:35.361109 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.355256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57892 (local address 127.16.71.129:41425)
0811 00:24:35.355427 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.355432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.355500 (+ 68us) server_negotiation.cc:407] Connection header received
0811 00:24:35.355706 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.355710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.355814 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.355969 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:24:35.356376 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.357200 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.358700 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.359143 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.359255 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.359892 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:24:35.360465 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:24:35.360600 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.360788 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.360870 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 00:24:35.370383 20660 raft_consensus.cc:1261] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 1 FOLLOWER]: Refusing update from remote peer f4c109f5550c48838a09dddaf5b6f2dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:35.370378 20431 raft_consensus.cc:1261] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 1 FOLLOWER]: Refusing update from remote peer f4c109f5550c48838a09dddaf5b6f2dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:24:35.371634 20794 consensus_queue.cc:1035] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c4f8ca786d4b91b78f8803290c78cf" member_type: VOTER last_known_addr { host: "127.16.71.131" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:35.372136 20787 consensus_queue.cc:1035] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cea23ae3a864926a81ecb818be53a01" member_type: VOTER last_known_addr { host: "127.16.71.130" port: 45659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:24:35.388357 20800 mvcc.cc:205] Tried to move back new op lower bound from 7317126657504845824 to 7317126657322430464. Current Snapshot: MvccSnapshot[applied={T|T < 7317126657504845824}]
I20260811 00:24:35.393975 20802 mvcc.cc:205] Tried to move back new op lower bound from 7317126657504845824 to 7317126657322430464. Current Snapshot: MvccSnapshot[applied={T|T < 7317126657504845824}]
I20260811 00:24:35.405355 19926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:35.405733 19926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:24:35.410399 19926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e96e91c92a7b47fcba804670551bc14d: Sending DeleteTablet for 3 replicas of tablet 4adc47142f3845698e531258d598f9ef
I20260811 00:24:35.411983 20160 tablet_service.cc:1514] Processing DeleteTablet for tablet 4adc47142f3845698e531258d598f9ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:35 UTC) from {username='slave'} at 127.0.0.1:57886
I20260811 00:24:35.412082 20397 tablet_service.cc:1514] Processing DeleteTablet for tablet 4adc47142f3845698e531258d598f9ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:35 UTC) from {username='slave'} at 127.0.0.1:44172
I20260811 00:24:35.412084 20626 tablet_service.cc:1514] Processing DeleteTablet for tablet 4adc47142f3845698e531258d598f9ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:24:35 UTC) from {username='slave'} at 127.0.0.1:52516
I20260811 00:24:35.420979 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20039
I20260811 00:24:35.446735 20825 tablet_replica.cc:330] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: stopping tablet replica
I20260811 00:24:35.447384 20825 raft_consensus.cc:2229] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:24:35.448117 20827 tablet_replica.cc:330] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: stopping tablet replica
I20260811 00:24:35.448216 20825 raft_consensus.cc:2258] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:35.448860 20827 raft_consensus.cc:2229] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:35.449568 20827 raft_consensus.cc:2258] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:35.450842 20826 tablet_replica.cc:330] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: stopping tablet replica
I20260811 00:24:35.451396 20826 raft_consensus.cc:2229] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:24:35.452025 20826 raft_consensus.cc:2258] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:24:35.452363 20827 ts_tablet_manager.cc:1905] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:35.454540 20826 ts_tablet_manager.cc:1905] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:35.456441 20825 ts_tablet_manager.cc:1905] T 4adc47142f3845698e531258d598f9ef P f4c109f5550c48838a09dddaf5b6f2dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:24:35.459671 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.424169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.16.71.129:41425)
0811 00:24:35.424855 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.424862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.444313 (+ 19451us) server_negotiation.cc:407] Connection header received
0811 00:24:35.444963 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.444971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.445127 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.445213 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:35.448790 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.450073 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.452214 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.452981 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.453115 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.457650 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:35.457666 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:35.457687 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:35.458566 (+ 879us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:35.458645 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:35.458679 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:35.458797 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.459121 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.459266 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392}
I20260811 00:24:35.460356 20827 ts_tablet_manager.cc:1918] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:35.460894 20827 log.cc:1196] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-2/wal/wals/4adc47142f3845698e531258d598f9ef
I20260811 00:24:35.461160 20271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:35.461555 20827 ts_tablet_manager.cc:1939] T 4adc47142f3845698e531258d598f9ef P 44c4f8ca786d4b91b78f8803290c78cf: Deleting consensus metadata
I20260811 00:24:35.463763 20826 ts_tablet_manager.cc:1918] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:24:35.463825 19904 catalog_manager.cc:4841] TS 44c4f8ca786d4b91b78f8803290c78cf (127.16.71.131:45661): tablet 4adc47142f3845698e531258d598f9ef (table test-table [id=dadb6a2894c34cb69a0b52a2820d52d2]) successfully deleted
I20260811 00:24:35.463979 20826 log.cc:1196] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Deleting WAL directory at /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407527335556-16670-0/minicluster-data/ts-1/wal/wals/4adc47142f3845698e531258d598f9ef
I20260811 00:24:35.464497 20826 ts_tablet_manager.cc:1939] T 4adc47142f3845698e531258d598f9ef P 6cea23ae3a864926a81ecb818be53a01: Deleting consensus metadata
I20260811 00:24:35.466771 19905 catalog_manager.cc:4841] TS 6cea23ae3a864926a81ecb818be53a01 (127.16.71.130:45659): tablet 4adc47142f3845698e531258d598f9ef (table test-table [id=dadb6a2894c34cb69a0b52a2820d52d2]) successfully deleted
I20260811 00:24:35.569164 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 20039
I20260811 00:24:35.594269 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20290
W20260811 00:24:35.594340 19906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.71.129:41425 (error 108)
I20260811 00:24:35.596333 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.595794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57908)
0811 00:24:35.595992 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.596161 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:24:35.596644 19906 catalog_manager.cc:4584] TS f4c109f5550c48838a09dddaf5b6f2dc (127.16.71.129:41425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4adc47142f3845698e531258d598f9ef: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
I20260811 00:24:35.602085 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.595251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.16.71.130:45659)
0811 00:24:35.595445 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.595452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.595550 (+ 98us) server_negotiation.cc:407] Connection header received
0811 00:24:35.595734 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.595740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.595840 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.595950 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:35.596373 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.597341 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.598687 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.599207 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.599322 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.600558 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:35.600568 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:35.600585 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:35.601307 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:35.601361 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:35.601383 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:35.601478 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.601627 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.601733 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:24:35.602890 20509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:35.650738 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.650154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57918)
0811 00:24:35.650434 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.650570 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 00:24:35.652065 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.651656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57934)
0811 00:24:35.651809 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.651899 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 00:24:35.696782 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 20290
I20260811 00:24:35.713052 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.712412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57940)
0811 00:24:35.712689 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.712822 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 00:24:35.714191 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.713783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57954)
0811 00:24:35.713937 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.714029 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 00:24:35.729352 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20521
I20260811 00:24:35.736558 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.730179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.16.71.131:45661)
0811 00:24:35.730376 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:35.730381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:35.730424 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:24:35.730588 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:35.730593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:35.730699 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:35.730794 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:35.731269 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.732004 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.733444 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:35.733884 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:35.733985 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:35.735106 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:35.735115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:35.735132 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:35.735758 (+ 626us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:35.735819 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:35.735847 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:35.735938 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:35.736082 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 00:24:35.736185 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:24:35.737298 20724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:35.795104 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.794581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57962)
0811 00:24:35.794820 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.794914 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 00:24:35.796473 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:35.795996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.129:41425 (local address 127.0.0.1:57964)
0811 00:24:35.796256 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 00:24:35.796348 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.129:41425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 00:24:35.828022 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 20521
I20260811 00:24:35.851176 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 19870
I20260811 00:24:35.852377 19987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:35.966260 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 19870
2026-08-11T00:24:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3198 ms)
[----------] 6 tests from EncryptionPolicyTest (24647 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:36 dist-test-slave-f7th krb5kdc[20839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:36 dist-test-slave-f7th krb5kdc[20839](info): set up 2 sockets
Aug 11 00:24:36 dist-test-slave-f7th krb5kdc[20839](info): commencing operation
krb5kdc: starting...
W20260811 00:24:37.310765 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:37 dist-test-slave-f7th krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:37.668761 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:38.045130 20860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:38.045454 20860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:38.045573 20860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:38.053424 20860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:38.053563 20860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:38.053617 20860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:38.053709 20860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:38.053769 20860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:38.064674 20860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:38.066642 20860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:38.068284 20860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:38.081948 20865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:38.081907 20868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:38.081982 20866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:38.082785 20860 server_base.cc:1034] running on GCE node
I20260811 00:24:38.083758 20860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:38.085531 20860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:38.086827 20860 hybrid_clock.cc:648] HybridClock initialized: now 1786407878086752 us; error 55 us; skew 500 ppm
I20260811 00:24:38.087294 20860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:38 dist-test-slave-f7th krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407878, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:38.095657 20860 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:38.097780 20860 webserver.cc:460] Webserver started at http://127.0.0.1:41149/ using document root <none> and password file <none>
I20260811 00:24:38.098610 20860 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:38.098762 20860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:38.099139 20860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:38.102598 20860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3"
format_stamp: "Formatted at 2026-08-11 00:24:38 on dist-test-slave-f7th"
server_key: "55373ef00bc5495230ce65d79ede4acd"
server_key_iv: "579c8723672bd09e8136e12cb183f9a5"
server_key_version: "encryptionkey@0"
I20260811 00:24:38.103536 20860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3"
format_stamp: "Formatted at 2026-08-11 00:24:38 on dist-test-slave-f7th"
server_key: "55373ef00bc5495230ce65d79ede4acd"
server_key_iv: "579c8723672bd09e8136e12cb183f9a5"
server_key_version: "encryptionkey@0"
I20260811 00:24:38.110047 20860 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:24:38.114331 20878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:38.116158 20860 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:24:38.116390 20860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3"
format_stamp: "Formatted at 2026-08-11 00:24:38 on dist-test-slave-f7th"
server_key: "55373ef00bc5495230ce65d79ede4acd"
server_key_iv: "579c8723672bd09e8136e12cb183f9a5"
server_key_version: "encryptionkey@0"
I20260811 00:24:38.116604 20860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:38.145525 20860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:38.150585 20860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:38.150856 20860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:38.170485 20860 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44525
I20260811 00:24:38.170532 20963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44525 every 8 connection(s)
I20260811 00:24:38.171955 20860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:38.175287 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 20860
I20260811 00:24:38.175487 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:38.176151 16670 external_mini_cluster.cc:1428] Setting key 7f1d14da21ef63781ae44ffdb4f460e7
I20260811 00:24:38.176415 20966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:38.191555 20966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3: Bootstrap starting.
Aug 11 00:24:38 dist-test-slave-f7th krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:24:38.195441 20966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:38.197120 20966 log.cc:824] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:38.200564 20966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3: No bootstrap required, opened a new log
I20260811 00:24:38.204262 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:38.178905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35903 (local address 127.0.0.1:44525)
0811 00:24:38.179890 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:38.179908 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:38.179960 (+ 52us) server_negotiation.cc:407] Connection header received
0811 00:24:38.182258 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:38.182290 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:38.182888 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:38.183392 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:38.185197 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:38.186656 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:38.188280 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:38.189241 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:38.189496 (+ 255us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:38.194522 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:38.194534 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:38.194554 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:38.194612 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:38.198622 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:38.199718 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:38.199727 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:38.199735 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:38.199980 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:38.200570 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:38.200574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:38.200576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:38.201315 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:38.201709 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:38.202501 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 00:24:38.202968 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":524,"threads_started":1}
I20260811 00:24:38.209829 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:38.210474 20966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44525 } }
I20260811 00:24:38.211073 20966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:38.211193 20966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3eb99d33bd4e688aed0aaaf017bfe3, State: Initialized, Role: FOLLOWER
I20260811 00:24:38.211876 20966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44525 } }
I20260811 00:24:38.212155 20966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:38.212298 20966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:38.212471 20966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:38.215106 20966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44525 } }
I20260811 00:24:38.215838 20966 leader_election.cc:302] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a3eb99d33bd4e688aed0aaaf017bfe3; no voters:
I20260811 00:24:38.216660 20966 leader_election.cc:288] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:38.216779 20994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:38.219127 20966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:38.219239 20994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [term 1 LEADER]: Becoming Leader. State: Replica: 3a3eb99d33bd4e688aed0aaaf017bfe3, State: Running, Role: LEADER
I20260811 00:24:38.219923 20994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44525 } }
I20260811 00:24:38.223351 20995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a3eb99d33bd4e688aed0aaaf017bfe3. Latest consensus state: current_term: 1 leader_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44525 } } }
I20260811 00:24:38.223858 20995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:38.227139 20998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3eb99d33bd4e688aed0aaaf017bfe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44525 } } }
I20260811 00:24:38.227687 20998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:38.237080 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:38.215566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38811 (local address 127.0.0.1:44525)
0811 00:24:38.215784 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:38.215791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:38.216233 (+ 442us) server_negotiation.cc:407] Connection header received
0811 00:24:38.216419 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:38.216426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:38.216523 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:38.216696 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:38.218935 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:38.220177 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:38.222225 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:38.222956 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:38.223121 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:38.227149 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:38.227157 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:38.227163 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:38.227220 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:38.231596 (+ 4376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:38.233146 (+ 1550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:38.233153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:38.233157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:38.233355 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:38.234037 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:38.234044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:38.234047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:38.234509 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:38.236014 (+ 1505us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:38.236226 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 00:24:38.236558 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:24:38.239863 21007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:38.246085 21007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:38.256836 21007 catalog_manager.cc:1347] Generated new cluster ID: 9bfa55338397435d8138906037ccab3b
I20260811 00:24:38.256991 21007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:38.270943 21007 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:38.273058 21007 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:38.288867 21007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a3eb99d33bd4e688aed0aaaf017bfe3: Generated new TSK 0
I20260811 00:24:38.289906 21007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:38.304816 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 20860
I20260811 00:24:38.306636 20959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:38.403431 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 20860
W20260811 00:24:38.426954 20836 connection.cc:535] client connection to 127.0.0.1:44525 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44525): Connection reset by peer (error 104)
2026-08-11T00:24:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:38 dist-test-slave-f7th krb5kdc[21030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:38 dist-test-slave-f7th krb5kdc[21030](info): set up 2 sockets
Aug 11 00:24:38 dist-test-slave-f7th krb5kdc[21030](info): commencing operation
krb5kdc: starting...
W20260811 00:24:39.729321 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:39 dist-test-slave-f7th krb5kdc[21030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:40.088976 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:40.456223 21048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:40.456605 21048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:40.456881 21048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:40.464697 21048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:40.464844 21048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:40.464938 21048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:40.465020 21048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:40.465085 21048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:40.475688 21048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:40.477177 21048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:40.478633 21048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:40.491751 21058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:40.491700 21055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:40.491784 21056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:40.492645 21048 server_base.cc:1034] running on GCE node
I20260811 00:24:40.493552 21048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:40.495218 21048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:40.496474 21048 hybrid_clock.cc:648] HybridClock initialized: now 1786407880496419 us; error 50 us; skew 500 ppm
I20260811 00:24:40.496909 21048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:40 dist-test-slave-f7th krb5kdc[21030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407880, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:40.500300 21048 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:40.502035 21048 webserver.cc:460] Webserver started at http://127.0.0.1:39491/ using document root <none> and password file <none>
I20260811 00:24:40.502826 21048 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:40.502955 21048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:40.503253 21048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:40.506503 21048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "49455e3fa1cc4092b525b38baeefc588"
format_stamp: "Formatted at 2026-08-11 00:24:40 on dist-test-slave-f7th"
server_key: "f7965bb6e41eb1d93adf7d3f02b7317f"
server_key_iv: "1600790eaa8a527eb31af58b67f09ea5"
server_key_version: "encryptionkey@0"
I20260811 00:24:40.507326 21048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "49455e3fa1cc4092b525b38baeefc588"
format_stamp: "Formatted at 2026-08-11 00:24:40 on dist-test-slave-f7th"
server_key: "f7965bb6e41eb1d93adf7d3f02b7317f"
server_key_iv: "1600790eaa8a527eb31af58b67f09ea5"
server_key_version: "encryptionkey@0"
I20260811 00:24:40.513374 21048 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:24:40.517283 21074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:40.518879 21048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:24:40.519105 21048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "49455e3fa1cc4092b525b38baeefc588"
format_stamp: "Formatted at 2026-08-11 00:24:40 on dist-test-slave-f7th"
server_key: "f7965bb6e41eb1d93adf7d3f02b7317f"
server_key_iv: "1600790eaa8a527eb31af58b67f09ea5"
server_key_version: "encryptionkey@0"
I20260811 00:24:40.519325 21048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:40.548050 21048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:40.552888 21048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:40.553215 21048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:40.572314 21048 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40591
I20260811 00:24:40.572283 21162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40591 every 8 connection(s)
I20260811 00:24:40.573988 21048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:40.574550 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21048
I20260811 00:24:40.574739 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:40.575318 16670 external_mini_cluster.cc:1428] Setting key ddbc719cce349bf310f55715289d1b55
I20260811 00:24:40.578826 21165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:40.589025 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:40.577350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59353 (local address 127.0.0.1:40591)
0811 00:24:40.578243 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:40.578256 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:40.578304 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:24:40.580036 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:40.580061 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:40.580457 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:40.580886 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:40.581456 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:40.582804 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:40.584242 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:40.585015 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:40.585219 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:40.586509 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:40.586517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:40.586532 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:40.587156 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:40.587238 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:40.587268 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:40.587433 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:40.587752 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 00:24:40.587936 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":417,"threads_started":1}
I20260811 00:24:40.590648 21165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588: Bootstrap starting.
I20260811 00:24:40.593384 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:40.594170 21165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:40.595232 21165 log.cc:824] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:40.598834 21165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588: No bootstrap required, opened a new log
I20260811 00:24:40.606016 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:40.598171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35139 (local address 127.0.0.1:40591)
0811 00:24:40.598391 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:40.598399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:40.598894 (+ 495us) server_negotiation.cc:407] Connection header received
0811 00:24:40.599168 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:40.599173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:40.599242 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:40.599335 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:40.600015 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:40.600930 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:40.603003 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:40.603530 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:40.603640 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:40.605060 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:40.605065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:40.605069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:40.605094 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:40.605140 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:40.605158 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:40.605260 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:40.605484 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 00:24:40.605539 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:24:40.609280 21165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49455e3fa1cc4092b525b38baeefc588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260811 00:24:40.609740 21165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:40.609859 21165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49455e3fa1cc4092b525b38baeefc588, State: Initialized, Role: FOLLOWER
I20260811 00:24:40.610620 21165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49455e3fa1cc4092b525b38baeefc588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260811 00:24:40.610883 21165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:40.611029 21165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:40.611228 21165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:40.613952 21165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49455e3fa1cc4092b525b38baeefc588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260811 00:24:40.614590 21165 leader_election.cc:302] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49455e3fa1cc4092b525b38baeefc588; no voters:
I20260811 00:24:40.615218 21165 leader_election.cc:288] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:40.615358 21188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:40.616070 21188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [term 1 LEADER]: Becoming Leader. State: Replica: 49455e3fa1cc4092b525b38baeefc588, State: Running, Role: LEADER
I20260811 00:24:40.616824 21188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49455e3fa1cc4092b525b38baeefc588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260811 00:24:40.617686 21165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:40.620029 21189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49455e3fa1cc4092b525b38baeefc588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49455e3fa1cc4092b525b38baeefc588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } } }
I20260811 00:24:40.619938 21190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49455e3fa1cc4092b525b38baeefc588. Latest consensus state: current_term: 1 leader_uuid: "49455e3fa1cc4092b525b38baeefc588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49455e3fa1cc4092b525b38baeefc588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } } }
I20260811 00:24:40.620354 21189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:40.620355 21190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588 [sys.catalog]: This master's current role is: LEADER
W20260811 00:24:40.635186 21228 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:24:40.635341 21228 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:24:40.635489 21199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:40.637960 21199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:40.648005 21199 catalog_manager.cc:1347] Generated new cluster ID: 49a52656b4634c9889c65003cda361c2
I20260811 00:24:40.648147 21199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:40.686108 21199 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:40.687674 21199 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:40.701062 21199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49455e3fa1cc4092b525b38baeefc588: Generated new TSK 0
I20260811 00:24:40.702272 21199 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:40.723156 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21048
W20260811 00:24:40.724850 21153 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59353
W20260811 00:24:40.725857 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21048: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:24:40.726082 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21048
W20260811 00:24:40.763095 21026 connection.cc:535] client connection to 127.0.0.1:40591 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40591): Connection reset by peer (error 104)
2026-08-11T00:24:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:40 dist-test-slave-f7th krb5kdc[21237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:40 dist-test-slave-f7th krb5kdc[21237](info): set up 2 sockets
Aug 11 00:24:40 dist-test-slave-f7th krb5kdc[21237](info): commencing operation
krb5kdc: starting...
W20260811 00:24:42.075192 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:42 dist-test-slave-f7th krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:42.425335 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43197
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:42.801630 21257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:42.801985 21257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:42.802115 21257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:42.809916 21257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:42.810074 21257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:42.810155 21257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:42.810242 21257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:42.810302 21257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:42.821436 21257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43197
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:42.822880 21257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:42.824306 21257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:42.837417 21265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:42.837451 21266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:42.837504 21272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:42.839012 21257 server_base.cc:1034] running on GCE node
I20260811 00:24:42.840718 21257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:42.843024 21257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:42.844378 21257 hybrid_clock.cc:648] HybridClock initialized: now 1786407882844269 us; error 74 us; skew 500 ppm
I20260811 00:24:42.844938 21257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:42 dist-test-slave-f7th krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:42.849057 21257 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:42.851131 21257 webserver.cc:460] Webserver started at http://127.0.0.1:37539/ using document root <none> and password file <none>
I20260811 00:24:42.851965 21257 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:42.852115 21257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:42.852481 21257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:42.856087 21257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "42e30b1b02fe4890a261231652f494bb"
format_stamp: "Formatted at 2026-08-11 00:24:42 on dist-test-slave-f7th"
server_key: "5cea0bff9de8873686eadf0a21daf9de"
server_key_iv: "73a9920c5233b7824a422c4ab3b6df81"
server_key_version: "encryptionkey@0"
I20260811 00:24:42.857110 21257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "42e30b1b02fe4890a261231652f494bb"
format_stamp: "Formatted at 2026-08-11 00:24:42 on dist-test-slave-f7th"
server_key: "5cea0bff9de8873686eadf0a21daf9de"
server_key_iv: "73a9920c5233b7824a422c4ab3b6df81"
server_key_version: "encryptionkey@0"
I20260811 00:24:42.863662 21257 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:42.868191 21281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:42.869973 21257 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:24:42.870195 21257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "42e30b1b02fe4890a261231652f494bb"
format_stamp: "Formatted at 2026-08-11 00:24:42 on dist-test-slave-f7th"
server_key: "5cea0bff9de8873686eadf0a21daf9de"
server_key_iv: "73a9920c5233b7824a422c4ab3b6df81"
server_key_version: "encryptionkey@0"
I20260811 00:24:42.870426 21257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:42.877074 21257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:42.881870 21257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:42.882184 21257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:42.902014 21257 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43197
I20260811 00:24:42.902064 21379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43197 every 8 connection(s)
I20260811 00:24:42.903630 21257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:42.907866 21381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:42.911373 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21257
I20260811 00:24:42.911585 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:42.912153 16670 external_mini_cluster.cc:1428] Setting key 76c021d5b7c2ad1cacc0f5200bf0d3f4
I20260811 00:24:42.921394 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:42.914253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52943 (local address 127.0.0.1:43197)
0811 00:24:42.915100 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:42.915113 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:42.915159 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:24:42.917336 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:42.917370 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:42.917860 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:42.918298 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:42.919533 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:42.919540 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:42.919556 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:42.919578 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:42.919644 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:42.919671 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:42.919859 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:42.919975 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 00:24:42.920256 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":422,"threads_started":1}
I20260811 00:24:42.921774 21381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb: Bootstrap starting.
I20260811 00:24:42.925302 21381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:42.925580 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:42.926728 21381 log.cc:824] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:42.930902 21381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb: No bootstrap required, opened a new log
I20260811 00:24:42.933996 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:42.931151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42895 (local address 127.0.0.1:43197)
0811 00:24:42.931366 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:42.931372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:42.931810 (+ 438us) server_negotiation.cc:407] Connection header received
0811 00:24:42.931988 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:42.931994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:42.932060 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:42.932182 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:42.933204 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:42.933211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:42.933215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:42.933237 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:42.933296 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:42.933318 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:42.933467 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:42.933606 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 00:24:42.933683 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:24:42.940179 21381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e30b1b02fe4890a261231652f494bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43197 } }
I20260811 00:24:42.940697 21381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:42.940791 21381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e30b1b02fe4890a261231652f494bb, State: Initialized, Role: FOLLOWER
I20260811 00:24:42.941491 21381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e30b1b02fe4890a261231652f494bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43197 } }
I20260811 00:24:42.941855 21381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:42.942106 21381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:42.942291 21381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:42.944893 21381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e30b1b02fe4890a261231652f494bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43197 } }
I20260811 00:24:42.945544 21381 leader_election.cc:302] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42e30b1b02fe4890a261231652f494bb; no voters:
I20260811 00:24:42.946331 21381 leader_election.cc:288] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:42.946508 21398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:42.949019 21381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:42.949107 21398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [term 1 LEADER]: Becoming Leader. State: Replica: 42e30b1b02fe4890a261231652f494bb, State: Running, Role: LEADER
I20260811 00:24:42.949905 21398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e30b1b02fe4890a261231652f494bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43197 } }
I20260811 00:24:42.953687 21400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42e30b1b02fe4890a261231652f494bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e30b1b02fe4890a261231652f494bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43197 } } }
I20260811 00:24:42.953622 21401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42e30b1b02fe4890a261231652f494bb. Latest consensus state: current_term: 1 leader_uuid: "42e30b1b02fe4890a261231652f494bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e30b1b02fe4890a261231652f494bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43197 } } }
I20260811 00:24:42.954205 21400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:42.954214 21401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:42.956524 21422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:42.962695 21422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:42.973696 21422 catalog_manager.cc:1347] Generated new cluster ID: 52ff65739bda4cff9b1e84954b46a4f1
I20260811 00:24:42.973848 21422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:42.998147 21422 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:42.998821 21422 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:43.008527 21422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42e30b1b02fe4890a261231652f494bb: Generated new TSK 0
I20260811 00:24:43.009667 21422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:43.023794 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21257
W20260811 00:24:43.025070 21372 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52943
W20260811 00:24:43.025812 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21257: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:24:43.025990 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21257
2026-08-11T00:24:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:43 dist-test-slave-f7th krb5kdc[21441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:43 dist-test-slave-f7th krb5kdc[21441](info): set up 2 sockets
Aug 11 00:24:43 dist-test-slave-f7th krb5kdc[21441](info): commencing operation
krb5kdc: starting...
W20260811 00:24:44.363590 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:44 dist-test-slave-f7th krb5kdc[21441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:44.716709 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:45.089118 21468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:45.089452 21468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:45.089579 21468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:45.097715 21468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:45.097891 21468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:45.097960 21468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:45.098055 21468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:45.098122 21468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:45.108839 21468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:45.110272 21468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:45.111699 21468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:45.124367 21479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:45.124347 21481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:45.124737 21484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:45.125345 21468 server_base.cc:1034] running on GCE node
I20260811 00:24:45.126281 21468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:45.128003 21468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:45.129264 21468 hybrid_clock.cc:648] HybridClock initialized: now 1786407885129207 us; error 50 us; skew 500 ppm
I20260811 00:24:45.129685 21468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:45 dist-test-slave-f7th krb5kdc[21441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407885, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:45.133458 21468 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:45.135321 21468 webserver.cc:460] Webserver started at http://127.0.0.1:38693/ using document root <none> and password file <none>
I20260811 00:24:45.136090 21468 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:45.136226 21468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:45.136543 21468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:45.139885 21468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "aed3b50ed2324a6fa6b4531ac360db50"
format_stamp: "Formatted at 2026-08-11 00:24:45 on dist-test-slave-f7th"
server_key: "a98b51da5b1ca99dd9cb224838111e79"
server_key_iv: "9b7ad813bc64661e6944bca097b55143"
server_key_version: "encryptionkey@0"
I20260811 00:24:45.140841 21468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "aed3b50ed2324a6fa6b4531ac360db50"
format_stamp: "Formatted at 2026-08-11 00:24:45 on dist-test-slave-f7th"
server_key: "a98b51da5b1ca99dd9cb224838111e79"
server_key_iv: "9b7ad813bc64661e6944bca097b55143"
server_key_version: "encryptionkey@0"
I20260811 00:24:45.146760 21468 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:24:45.151118 21493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:45.152936 21468 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:24:45.153155 21468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "aed3b50ed2324a6fa6b4531ac360db50"
format_stamp: "Formatted at 2026-08-11 00:24:45 on dist-test-slave-f7th"
server_key: "a98b51da5b1ca99dd9cb224838111e79"
server_key_iv: "9b7ad813bc64661e6944bca097b55143"
server_key_version: "encryptionkey@0"
I20260811 00:24:45.153385 21468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:45.189687 21468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:45.194686 21468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:45.195010 21468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:45.214864 21468 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46329
I20260811 00:24:45.214896 21588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46329 every 8 connection(s)
I20260811 00:24:45.216394 21468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:45.220738 21590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:45.223579 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21468
I20260811 00:24:45.223801 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:45.224323 16670 external_mini_cluster.cc:1428] Setting key 83a17bf0713683b7f3e10862123b3453
I20260811 00:24:45.235291 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50: Bootstrap starting.
I20260811 00:24:45.239662 21590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50: Neither blocks nor log segments found. Creating new log.
Aug 11 00:24:45 dist-test-slave-f7th krb5kdc[21441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:24:45.241295 21590 log.cc:824] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:45.245477 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50: No bootstrap required, opened a new log
I20260811 00:24:45.253566 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:45.226391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52305 (local address 127.0.0.1:46329)
0811 00:24:45.227359 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:45.227376 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:45.227441 (+ 65us) server_negotiation.cc:407] Connection header received
0811 00:24:45.229396 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:45.229427 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:45.229906 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:45.230263 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:45.232545 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:45.234062 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:45.235921 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:45.236701 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:45.236855 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:45.242254 (+ 5399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:45.242264 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:45.242282 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:45.242336 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:45.246628 (+ 4292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:45.247860 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:45.247870 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:45.247881 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:45.248156 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:45.248804 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:45.248810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:45.248813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:45.249718 (+ 905us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:45.250078 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:45.251259 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0811 00:24:45.251889 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":457,"threads_started":1}
I20260811 00:24:45.256059 21590 raft_consensus.cc:348] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed3b50ed2324a6fa6b4531ac360db50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46329 } }
I20260811 00:24:45.256682 21590 raft_consensus.cc:374] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:45.256819 21590 raft_consensus.cc:729] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3b50ed2324a6fa6b4531ac360db50, State: Initialized, Role: FOLLOWER
I20260811 00:24:45.257555 21590 consensus_queue.cc:260] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed3b50ed2324a6fa6b4531ac360db50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46329 } }
I20260811 00:24:45.257901 21590 raft_consensus.cc:388] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:45.258064 21590 raft_consensus.cc:482] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:45.258251 21590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:45.259584 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:45.261078 21590 raft_consensus.cc:504] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed3b50ed2324a6fa6b4531ac360db50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46329 } }
I20260811 00:24:45.261688 21590 leader_election.cc:302] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aed3b50ed2324a6fa6b4531ac360db50; no voters:
I20260811 00:24:45.262434 21590 leader_election.cc:288] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:45.262643 21597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:45.264820 21590 sys_catalog.cc:564] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:45.270260 21597 raft_consensus.cc:686] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [term 1 LEADER]: Becoming Leader. State: Replica: aed3b50ed2324a6fa6b4531ac360db50, State: Running, Role: LEADER
I20260811 00:24:45.271111 21597 consensus_queue.cc:237] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed3b50ed2324a6fa6b4531ac360db50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46329 } }
I20260811 00:24:45.274212 21601 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aed3b50ed2324a6fa6b4531ac360db50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed3b50ed2324a6fa6b4531ac360db50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46329 } } }
I20260811 00:24:45.274660 21601 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:45.275779 21630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:45.275615 21604 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aed3b50ed2324a6fa6b4531ac360db50. Latest consensus state: current_term: 1 leader_uuid: "aed3b50ed2324a6fa6b4531ac360db50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed3b50ed2324a6fa6b4531ac360db50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46329 } } }
I20260811 00:24:45.275956 21604 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:45.282940 21630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:45.291975 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:45.272531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36765 (local address 127.0.0.1:46329)
0811 00:24:45.272764 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:45.272771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:45.273186 (+ 415us) server_negotiation.cc:407] Connection header received
0811 00:24:45.273409 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:45.273413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:45.273496 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:45.273590 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:45.276375 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:45.277510 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:45.279872 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:45.280610 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:45.280807 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:45.284102 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:45.284110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:45.284116 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:45.284163 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:45.288310 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:45.289423 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:45.289429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:45.289433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:45.289615 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:45.290172 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:45.290177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:45.290179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:45.290540 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:45.290777 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:45.291295 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 00:24:45.291578 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:24:45.293577 21630 catalog_manager.cc:1347] Generated new cluster ID: 43c45e8f2c55482bb89bdae36833f1dc
I20260811 00:24:45.293751 21630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:45.308522 21630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:45.310048 21630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:45.326596 21630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aed3b50ed2324a6fa6b4531ac360db50: Generated new TSK 0
I20260811 00:24:45.327564 21630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:45.336802 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21468
I20260811 00:24:45.338014 21581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:45.432327 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21468
2026-08-11T00:24:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:45 dist-test-slave-f7th krb5kdc[21646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:45 dist-test-slave-f7th krb5kdc[21646](info): set up 2 sockets
Aug 11 00:24:45 dist-test-slave-f7th krb5kdc[21646](info): commencing operation
krb5kdc: starting...
W20260811 00:24:46.760982 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.003s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:46 dist-test-slave-f7th krb5kdc[21646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:47.132987 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:47.502472 21666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:47.502811 21666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:47.502907 21666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:47.510665 21666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:47.510811 21666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:47.510861 21666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:47.510922 21666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:47.510967 21666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:47.522202 21666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:47.523630 21666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:47.525105 21666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:47.538391 21679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:47.538477 21681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:47.538443 21678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:47.539461 21666 server_base.cc:1034] running on GCE node
I20260811 00:24:47.540403 21666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:47.542088 21666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:47.543347 21666 hybrid_clock.cc:648] HybridClock initialized: now 1786407887543288 us; error 51 us; skew 500 ppm
I20260811 00:24:47.543766 21666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:47 dist-test-slave-f7th krb5kdc[21646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:47.547449 21666 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:47.549299 21666 webserver.cc:460] Webserver started at http://127.0.0.1:33711/ using document root <none> and password file <none>
I20260811 00:24:47.550138 21666 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:47.550282 21666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:47.550637 21666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:47.554149 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "c8a0c1d949e344168fb29a1df70b01e8"
format_stamp: "Formatted at 2026-08-11 00:24:47 on dist-test-slave-f7th"
server_key: "73d4e512bf9e4fd15c2a68f291114836"
server_key_iv: "d7b06b3192d4c759276be7ab34978ca8"
server_key_version: "encryptionkey@0"
I20260811 00:24:47.555054 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "c8a0c1d949e344168fb29a1df70b01e8"
format_stamp: "Formatted at 2026-08-11 00:24:47 on dist-test-slave-f7th"
server_key: "73d4e512bf9e4fd15c2a68f291114836"
server_key_iv: "d7b06b3192d4c759276be7ab34978ca8"
server_key_version: "encryptionkey@0"
I20260811 00:24:47.561478 21666 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:24:47.565923 21692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:47.567713 21666 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:24:47.567953 21666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "c8a0c1d949e344168fb29a1df70b01e8"
format_stamp: "Formatted at 2026-08-11 00:24:47 on dist-test-slave-f7th"
server_key: "73d4e512bf9e4fd15c2a68f291114836"
server_key_iv: "d7b06b3192d4c759276be7ab34978ca8"
server_key_version: "encryptionkey@0"
I20260811 00:24:47.568193 21666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:47.584133 21666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:47.589659 21666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:47.590024 21666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:47.609470 21666 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40449
I20260811 00:24:47.609546 21770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40449 every 8 connection(s)
I20260811 00:24:47.610966 21666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:47.615152 21771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:47.617810 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21666
I20260811 00:24:47.617995 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:47.618444 16670 external_mini_cluster.cc:1428] Setting key 59fecf3895b465fb760042d8bb3b621c
I20260811 00:24:47.631160 21771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8: Bootstrap starting.
I20260811 00:24:47.633556 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:47.620199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59641 (local address 127.0.0.1:40449)
0811 00:24:47.621124 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:47.621141 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:47.621192 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:24:47.622896 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:47.622926 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:47.623445 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:47.623831 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:47.624610 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:47.626134 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:47.628255 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:47.629175 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:47.629336 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:47.630753 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:47.630762 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:47.630781 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:47.631480 (+ 699us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:47.631576 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:47.631609 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:47.631709 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:47.632049 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 00:24:47.632375 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":449,"threads_started":1}
I20260811 00:24:47.634918 21771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:47.636108 21771 log.cc:824] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:47.637992 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:47.640002 21771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8: No bootstrap required, opened a new log
I20260811 00:24:47.650697 21771 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0c1d949e344168fb29a1df70b01e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40449 } }
I20260811 00:24:47.651235 21771 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:47.651422 21771 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a0c1d949e344168fb29a1df70b01e8, State: Initialized, Role: FOLLOWER
I20260811 00:24:47.652438 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:47.643562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57799 (local address 127.0.0.1:40449)
0811 00:24:47.643767 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:47.643774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:47.644177 (+ 403us) server_negotiation.cc:407] Connection header received
0811 00:24:47.644348 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:47.644354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:47.644447 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:47.644583 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:47.645168 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:47.646324 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:47.648117 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:47.648945 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:47.649124 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:47.651417 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:47.651425 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:47.651431 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:47.651460 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:47.651526 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:47.651555 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:47.651629 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:47.651866 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 00:24:47.651983 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:24:47.652559 21771 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0c1d949e344168fb29a1df70b01e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40449 } }
I20260811 00:24:47.653077 21790 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:24:47.653374 21790 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0c1d949e344168fb29a1df70b01e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40449 } }
I20260811 00:24:47.654062 21790 leader_election.cc:302] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a0c1d949e344168fb29a1df70b01e8; no voters:
I20260811 00:24:47.653052 21771 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:47.655375 21771 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:47.655611 21771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:47.655691 21790 leader_election.cc:288] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260811 00:24:47.658329 21771 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0c1d949e344168fb29a1df70b01e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40449 } }
I20260811 00:24:47.658661 21771 leader_election.cc:302] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a0c1d949e344168fb29a1df70b01e8; no voters:
I20260811 00:24:47.658773 21791 raft_consensus.cc:2741] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 00:24:47.658921 21771 leader_election.cc:288] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:47.659307 21790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:47.660671 21790 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [term 1 LEADER]: Becoming Leader. State: Replica: c8a0c1d949e344168fb29a1df70b01e8, State: Running, Role: LEADER
I20260811 00:24:47.661312 21790 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0c1d949e344168fb29a1df70b01e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40449 } }
I20260811 00:24:47.662015 21771 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:47.664083 21791 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8a0c1d949e344168fb29a1df70b01e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0c1d949e344168fb29a1df70b01e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40449 } } }
I20260811 00:24:47.664064 21792 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a0c1d949e344168fb29a1df70b01e8. Latest consensus state: current_term: 1 leader_uuid: "c8a0c1d949e344168fb29a1df70b01e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0c1d949e344168fb29a1df70b01e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40449 } } }
I20260811 00:24:47.664932 21791 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:47.665015 21792 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:47.665922 21803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:47.672364 21803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:47.691476 21803 catalog_manager.cc:1347] Generated new cluster ID: ff35f81f097a42fbb2224d14bd93fff4
I20260811 00:24:47.691628 21803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:47.743597 21803 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:47.745541 21803 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:47.753975 21803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8a0c1d949e344168fb29a1df70b01e8: Generated new TSK 0
I20260811 00:24:47.754926 21803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:47.767637 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21666
W20260811 00:24:47.769116 21762 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59641
W20260811 00:24:47.770051 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21666: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:24:47.770223 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21666
2026-08-11T00:24:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:47 dist-test-slave-f7th krb5kdc[21828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:47 dist-test-slave-f7th krb5kdc[21828](info): set up 2 sockets
Aug 11 00:24:47 dist-test-slave-f7th krb5kdc[21828](info): commencing operation
krb5kdc: starting...
W20260811 00:24:49.112522 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:49 dist-test-slave-f7th krb5kdc[21828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:49.469632 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:49.855856 21853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:49.856252 21853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:49.856527 21853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:49.864866 21853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:49.865051 21853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:49.865129 21853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:49.865240 21853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:49.865300 21853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:49.876574 21853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:49.878062 21853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:49.879503 21853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:49.893292 21864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:49.893281 21862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:49.893946 21861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:49.898628 21853 server_base.cc:1034] running on GCE node
I20260811 00:24:49.899608 21853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:49.901324 21853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:49.902598 21853 hybrid_clock.cc:648] HybridClock initialized: now 1786407889902549 us; error 41 us; skew 500 ppm
I20260811 00:24:49.903017 21853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:49 dist-test-slave-f7th krb5kdc[21828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:49.910301 21853 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:49.912122 21853 webserver.cc:460] Webserver started at http://127.0.0.1:41949/ using document root <none> and password file <none>
I20260811 00:24:49.913002 21853 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:49.913130 21853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:49.913407 21853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:49.916857 21853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "73466fdef27c44be92585bd9a7be3908"
format_stamp: "Formatted at 2026-08-11 00:24:49 on dist-test-slave-f7th"
server_key: "e811da9774e59bd77340d06dcfe181c0"
server_key_iv: "ff4609f908903552873445612cc3fcbd"
server_key_version: "encryptionkey@0"
I20260811 00:24:49.917861 21853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "73466fdef27c44be92585bd9a7be3908"
format_stamp: "Formatted at 2026-08-11 00:24:49 on dist-test-slave-f7th"
server_key: "e811da9774e59bd77340d06dcfe181c0"
server_key_iv: "ff4609f908903552873445612cc3fcbd"
server_key_version: "encryptionkey@0"
I20260811 00:24:49.924705 21853 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:24:49.929185 21878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:49.931022 21853 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:24:49.931245 21853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "73466fdef27c44be92585bd9a7be3908"
format_stamp: "Formatted at 2026-08-11 00:24:49 on dist-test-slave-f7th"
server_key: "e811da9774e59bd77340d06dcfe181c0"
server_key_iv: "ff4609f908903552873445612cc3fcbd"
server_key_version: "encryptionkey@0"
I20260811 00:24:49.931483 21853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:49.938715 21853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:49.943681 21853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:49.944029 21853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:49.965646 21853 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33635
I20260811 00:24:49.965667 21952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33635 every 8 connection(s)
I20260811 00:24:49.967387 21853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:49.971951 21955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:49.977653 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 21853
I20260811 00:24:49.977846 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:49.978319 16670 external_mini_cluster.cc:1428] Setting key c23bf0bd5ecfb1fd596afa47e5cbabea
I20260811 00:24:49.983932 21955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908: Bootstrap starting.
I20260811 00:24:49.986733 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:49.980215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44099 (local address 127.0.0.1:33635)
0811 00:24:49.981148 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:49.981163 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:49.981216 (+ 53us) server_negotiation.cc:407] Connection header received
0811 00:24:49.983057 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:49.983085 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:49.983482 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:49.983869 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:49.985043 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:49.985050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:49.985065 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:49.985082 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:49.985150 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:49.985172 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:49.985351 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:49.985452 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 00:24:49.985714 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":449,"threads_started":1}
I20260811 00:24:49.987794 21955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:49.989027 21955 log.cc:824] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:49.991016 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:49.992853 21955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908: No bootstrap required, opened a new log
I20260811 00:24:49.999503 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:49.996769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41115 (local address 127.0.0.1:33635)
0811 00:24:49.996974 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:49.996978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:49.997227 (+ 249us) server_negotiation.cc:407] Connection header received
0811 00:24:49.997584 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:49.997591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:49.997653 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:49.997749 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:49.998789 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:49.998793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:49.998797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:49.998816 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:49.998866 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:49.998882 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:49.999024 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:49.999125 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 00:24:49.999177 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 00:24:50.002871 21955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73466fdef27c44be92585bd9a7be3908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260811 00:24:50.003341 21955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:50.003450 21955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73466fdef27c44be92585bd9a7be3908, State: Initialized, Role: FOLLOWER
I20260811 00:24:50.004132 21955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73466fdef27c44be92585bd9a7be3908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260811 00:24:50.004416 21955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:50.004563 21955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:50.004799 21955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:50.007282 21955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73466fdef27c44be92585bd9a7be3908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260811 00:24:50.007920 21955 leader_election.cc:302] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73466fdef27c44be92585bd9a7be3908; no voters:
I20260811 00:24:50.008601 21955 leader_election.cc:288] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:50.008821 21977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:50.009541 21977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [term 1 LEADER]: Becoming Leader. State: Replica: 73466fdef27c44be92585bd9a7be3908, State: Running, Role: LEADER
I20260811 00:24:50.010574 21977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73466fdef27c44be92585bd9a7be3908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260811 00:24:50.012290 21955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:50.013739 21979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73466fdef27c44be92585bd9a7be3908. Latest consensus state: current_term: 1 leader_uuid: "73466fdef27c44be92585bd9a7be3908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73466fdef27c44be92585bd9a7be3908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } } }
I20260811 00:24:50.013774 21980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73466fdef27c44be92585bd9a7be3908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73466fdef27c44be92585bd9a7be3908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } } }
I20260811 00:24:50.014286 21979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:50.014421 21980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:50.016907 21985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:50.023144 21985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:50.034219 21985 catalog_manager.cc:1347] Generated new cluster ID: 2d9dcc5741c74dbcb5afd5af598cdd53
I20260811 00:24:50.034368 21985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:50.060827 21985 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:50.061434 21985 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:50.075714 21985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73466fdef27c44be92585bd9a7be3908: Generated new TSK 0
I20260811 00:24:50.076793 21985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:50.092464 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21853
W20260811 00:24:50.093760 21946 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44099
W20260811 00:24:50.094605 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 21853: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:24:50.094762 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 21853
2026-08-11T00:24:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:50 dist-test-slave-f7th krb5kdc[22006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:50 dist-test-slave-f7th krb5kdc[22006](info): set up 2 sockets
Aug 11 00:24:50 dist-test-slave-f7th krb5kdc[22006](info): commencing operation
krb5kdc: starting...
W20260811 00:24:51.411232 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:51 dist-test-slave-f7th krb5kdc[22006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:51.769215 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:52.151434 22024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:52.151780 22024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:52.151925 22024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:52.160202 22024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:52.160347 22024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:52.160403 22024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:52.160497 22024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:52.160559 22024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:52.171469 22024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:52.172964 22024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:52.174422 22024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:52.187525 22035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:52.187558 22036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:52.187476 22039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:52.188727 22024 server_base.cc:1034] running on GCE node
I20260811 00:24:52.189689 22024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:52.191372 22024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:52.192683 22024 hybrid_clock.cc:648] HybridClock initialized: now 1786407892192607 us; error 43 us; skew 500 ppm
I20260811 00:24:52.193116 22024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:52 dist-test-slave-f7th krb5kdc[22006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:52.196874 22024 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:52.198678 22024 webserver.cc:460] Webserver started at http://127.0.0.1:45237/ using document root <none> and password file <none>
I20260811 00:24:52.199496 22024 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:52.199637 22024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:52.199970 22024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:52.203380 22024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "e032828db52e412888af08694b1ff9df"
format_stamp: "Formatted at 2026-08-11 00:24:52 on dist-test-slave-f7th"
server_key: "2ba6c4f88a2346d4ff2860c4e52af1de"
server_key_iv: "dca3ff1f63ce9653ab26eeec7bc0b85c"
server_key_version: "encryptionkey@0"
I20260811 00:24:52.204267 22024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "e032828db52e412888af08694b1ff9df"
format_stamp: "Formatted at 2026-08-11 00:24:52 on dist-test-slave-f7th"
server_key: "2ba6c4f88a2346d4ff2860c4e52af1de"
server_key_iv: "dca3ff1f63ce9653ab26eeec7bc0b85c"
server_key_version: "encryptionkey@0"
I20260811 00:24:52.210487 22024 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 00:24:52.214948 22052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:52.216603 22024 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 00:24:52.216934 22024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "e032828db52e412888af08694b1ff9df"
format_stamp: "Formatted at 2026-08-11 00:24:52 on dist-test-slave-f7th"
server_key: "2ba6c4f88a2346d4ff2860c4e52af1de"
server_key_iv: "dca3ff1f63ce9653ab26eeec7bc0b85c"
server_key_version: "encryptionkey@0"
I20260811 00:24:52.217156 22024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:52.232127 22024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:52.237459 22024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:52.237799 22024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:52.257889 22024 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35449
I20260811 00:24:52.257941 22138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35449 every 8 connection(s)
I20260811 00:24:52.259442 22024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:52.261708 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22024
I20260811 00:24:52.261909 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:52.262348 16670 external_mini_cluster.cc:1428] Setting key 018ceed2a0096cfed5024aeecf00dbf4
I20260811 00:24:52.263871 22139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:24:52 dist-test-slave-f7th krb5kdc[22006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:24:52.279006 22139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df: Bootstrap starting.
I20260811 00:24:52.283231 22139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:52.284554 22139 log.cc:824] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:52.288794 22139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df: No bootstrap required, opened a new log
I20260811 00:24:52.290037 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:52.264409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:43643 (local address 127.0.0.1:35449)
0811 00:24:52.265255 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:52.265267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:52.265311 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:24:52.266997 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:52.267024 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:52.267487 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:52.267906 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:52.269984 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:52.271662 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:52.273612 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:52.274438 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:52.274617 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:52.280324 (+ 5707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:52.280334 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:52.280351 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:52.280413 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:52.284244 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:52.285638 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:52.285650 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:52.285663 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:52.285953 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:52.286641 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:52.286646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:52.286648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:52.287316 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:52.287609 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:52.288342 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 00:24:52.288809 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":369,"threads_started":1}
I20260811 00:24:52.295089 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:52.299801 22139 raft_consensus.cc:348] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e032828db52e412888af08694b1ff9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35449 } }
I20260811 00:24:52.300442 22139 raft_consensus.cc:374] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:52.300580 22139 raft_consensus.cc:729] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e032828db52e412888af08694b1ff9df, State: Initialized, Role: FOLLOWER
I20260811 00:24:52.301424 22139 consensus_queue.cc:260] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e032828db52e412888af08694b1ff9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35449 } }
I20260811 00:24:52.301779 22139 raft_consensus.cc:388] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:52.301949 22139 raft_consensus.cc:482] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:52.302160 22139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:52.305565 22139 raft_consensus.cc:504] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e032828db52e412888af08694b1ff9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35449 } }
I20260811 00:24:52.306354 22139 leader_election.cc:302] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e032828db52e412888af08694b1ff9df; no voters:
I20260811 00:24:52.307169 22139 leader_election.cc:288] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:52.307291 22155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:52.309033 22155 raft_consensus.cc:686] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [term 1 LEADER]: Becoming Leader. State: Replica: e032828db52e412888af08694b1ff9df, State: Running, Role: LEADER
I20260811 00:24:52.309758 22155 consensus_queue.cc:237] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e032828db52e412888af08694b1ff9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35449 } }
I20260811 00:24:52.309942 22139 sys_catalog.cc:564] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:52.312681 22157 sys_catalog.cc:455] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e032828db52e412888af08694b1ff9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e032828db52e412888af08694b1ff9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35449 } } }
I20260811 00:24:52.313107 22157 sys_catalog.cc:458] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:52.315239 22158 sys_catalog.cc:455] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [sys.catalog]: SysCatalogTable state changed. Reason: New leader e032828db52e412888af08694b1ff9df. Latest consensus state: current_term: 1 leader_uuid: "e032828db52e412888af08694b1ff9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e032828db52e412888af08694b1ff9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35449 } } }
I20260811 00:24:52.315747 22158 sys_catalog.cc:458] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:52.317922 22162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:52.321004 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:52.301213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:38441 (local address 127.0.0.1:35449)
0811 00:24:52.301482 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:52.301491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:52.301992 (+ 501us) server_negotiation.cc:407] Connection header received
0811 00:24:52.302231 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:52.302235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:52.302325 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:52.302417 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:52.304501 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:52.305448 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:52.307479 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:52.308273 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:52.308452 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:52.312122 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:52.312130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:52.312136 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:52.312193 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:52.316373 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:52.317695 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:52.317701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:52.317704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:52.317933 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:52.318642 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:52.318648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:52.318651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:52.319158 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:52.319531 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:52.320107 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 00:24:52.320408 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:24:52.325186 22162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:52.337225 22162 catalog_manager.cc:1347] Generated new cluster ID: 225b3ab6a7c14edc81101dd466a60d17
I20260811 00:24:52.337379 22162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:52.365305 22162 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:52.367316 22162 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:52.391746 22162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e032828db52e412888af08694b1ff9df: Generated new TSK 0
I20260811 00:24:52.393024 22162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:52.408191 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22024
I20260811 00:24:52.409584 22134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:52.507125 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22024
W20260811 00:24:52.532150 22003 connection.cc:535] client connection to 127.0.0.1:35449 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35449): Connection reset by peer (error 104)
2026-08-11T00:24:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:52 dist-test-slave-f7th krb5kdc[22196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:52 dist-test-slave-f7th krb5kdc[22196](info): set up 2 sockets
Aug 11 00:24:52 dist-test-slave-f7th krb5kdc[22196](info): commencing operation
krb5kdc: starting...
W20260811 00:24:53.843001 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.001s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:54 dist-test-slave-f7th krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:54.200655 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:54.573263 22219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:54.573618 22219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:54.573766 22219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:54.581326 22219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:54.581471 22219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:54.581552 22219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:54.581640 22219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:54.581699 22219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:54.592423 22219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:54.593914 22219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:54.595305 22219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:54.607725 22226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:54.607757 22230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:54.607882 22227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:54.609015 22219 server_base.cc:1034] running on GCE node
I20260811 00:24:54.609917 22219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:54.611483 22219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:54.612772 22219 hybrid_clock.cc:648] HybridClock initialized: now 1786407894612710 us; error 41 us; skew 500 ppm
I20260811 00:24:54.613178 22219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:54 dist-test-slave-f7th krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:54.616688 22219 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:54.618486 22219 webserver.cc:460] Webserver started at http://127.0.0.1:33291/ using document root <none> and password file <none>
I20260811 00:24:54.619287 22219 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:54.619421 22219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:54.619710 22219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:54.623066 22219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "c694a9a4368a4387b8d9b64fabf5bdb8"
format_stamp: "Formatted at 2026-08-11 00:24:54 on dist-test-slave-f7th"
server_key: "2ac4d49e0e4a419410bd5fa196d87eb8"
server_key_iv: "a9757fa81055cf1f320803005f3793b7"
server_key_version: "encryptionkey@0"
I20260811 00:24:54.623900 22219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "c694a9a4368a4387b8d9b64fabf5bdb8"
format_stamp: "Formatted at 2026-08-11 00:24:54 on dist-test-slave-f7th"
server_key: "2ac4d49e0e4a419410bd5fa196d87eb8"
server_key_iv: "a9757fa81055cf1f320803005f3793b7"
server_key_version: "encryptionkey@0"
I20260811 00:24:54.629967 22219 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:24:54.634389 22246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:54.636303 22219 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:24:54.636526 22219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "c694a9a4368a4387b8d9b64fabf5bdb8"
format_stamp: "Formatted at 2026-08-11 00:24:54 on dist-test-slave-f7th"
server_key: "2ac4d49e0e4a419410bd5fa196d87eb8"
server_key_iv: "a9757fa81055cf1f320803005f3793b7"
server_key_version: "encryptionkey@0"
I20260811 00:24:54.636819 22219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:54.654245 22219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:54.659232 22219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:54.659509 22219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:54.679976 22219 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46649
I20260811 00:24:54.679999 22337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46649 every 8 connection(s)
I20260811 00:24:54.681646 22219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:54.685971 22339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:54.686945 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22219
I20260811 00:24:54.687152 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:54.687614 16670 external_mini_cluster.cc:1428] Setting key 00eefeb424606bbe3a97758bbcf25492
I20260811 00:24:54.702430 22339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8: Bootstrap starting.
I20260811 00:24:54.704602 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:54.689899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:60661 (local address 127.0.0.1:46649)
0811 00:24:54.690795 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:54.690811 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:54.690869 (+ 58us) server_negotiation.cc:407] Connection header received
0811 00:24:54.693395 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:54.693434 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:54.694116 (+ 682us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:54.694659 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:54.695213 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:54.697185 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:54.698741 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:54.699945 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:54.700149 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:54.701500 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:54.701512 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:54.701534 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:54.702302 (+ 768us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:54.702425 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:54.702464 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:54.702667 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:54.702976 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 00:24:54.703187 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":332,"threads_started":1}
I20260811 00:24:54.706507 22339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:54.707772 22339 log.cc:824] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:54.709251 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:54.711022 22339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8: No bootstrap required, opened a new log
I20260811 00:24:54.720539 22339 raft_consensus.cc:348] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46649 } }
I20260811 00:24:54.721132 22339 raft_consensus.cc:374] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:54.721212 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:54.713854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:60013 (local address 127.0.0.1:46649)
0811 00:24:54.714052 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:54.714058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:54.714374 (+ 316us) server_negotiation.cc:407] Connection header received
0811 00:24:54.714553 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:54.714559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:54.714649 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:54.714779 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:54.715389 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:54.716597 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:54.718193 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:54.718926 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:54.719065 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:54.720350 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:54.720356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:54.720361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:54.720386 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:54.720455 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:54.720477 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:54.720654 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:54.720833 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 00:24:54.720898 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:24:54.721279 22339 raft_consensus.cc:729] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c694a9a4368a4387b8d9b64fabf5bdb8, State: Initialized, Role: FOLLOWER
I20260811 00:24:54.722015 22339 consensus_queue.cc:260] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46649 } }
I20260811 00:24:54.722278 22339 raft_consensus.cc:388] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:54.722427 22339 raft_consensus.cc:482] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:54.722617 22339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:54.725273 22339 raft_consensus.cc:504] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46649 } }
I20260811 00:24:54.725850 22339 leader_election.cc:302] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c694a9a4368a4387b8d9b64fabf5bdb8; no voters:
I20260811 00:24:54.726511 22339 leader_election.cc:288] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:54.726665 22357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:54.727453 22357 raft_consensus.cc:686] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [term 1 LEADER]: Becoming Leader. State: Replica: c694a9a4368a4387b8d9b64fabf5bdb8, State: Running, Role: LEADER
I20260811 00:24:54.728052 22357 consensus_queue.cc:237] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46649 } }
I20260811 00:24:54.728863 22339 sys_catalog.cc:564] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:54.731248 22359 sys_catalog.cc:455] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c694a9a4368a4387b8d9b64fabf5bdb8. Latest consensus state: current_term: 1 leader_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46649 } } }
I20260811 00:24:54.731650 22359 sys_catalog.cc:458] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:54.732348 22358 sys_catalog.cc:455] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c694a9a4368a4387b8d9b64fabf5bdb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46649 } } }
I20260811 00:24:54.732685 22358 sys_catalog.cc:458] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:54.737602 22367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:54.743422 22367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:54.756000 22367 catalog_manager.cc:1347] Generated new cluster ID: 87a8a035ea494e66b0f39cf01001b916
I20260811 00:24:54.756240 22367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:54.771761 22367 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:54.773375 22367 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:54.784266 22367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c694a9a4368a4387b8d9b64fabf5bdb8: Generated new TSK 0
I20260811 00:24:54.785396 22367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:54.803289 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22219
W20260811 00:24:54.805353 22329 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:60661
W20260811 00:24:54.806222 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22219: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:24:54.806370 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22219
W20260811 00:24:54.839290 22192 connection.cc:535] client connection to 127.0.0.1:46649 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46649): Connection reset by peer (error 104)
2026-08-11T00:24:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:54 dist-test-slave-f7th krb5kdc[22403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:54 dist-test-slave-f7th krb5kdc[22403](info): set up 2 sockets
Aug 11 00:24:54 dist-test-slave-f7th krb5kdc[22403](info): commencing operation
krb5kdc: starting...
W20260811 00:24:56.138592 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:56 dist-test-slave-f7th krb5kdc[22403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:56.496804 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:56.880657 22423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:56.881042 22423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:56.881188 22423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:56.888904 22423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:56.889068 22423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:56.889153 22423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:56.889236 22423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:56.889295 22423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:56.900087 22423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:56.901659 22423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:56.903163 22423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:56.917001 22433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:56.916922 22434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:56.916882 22437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:56.918258 22423 server_base.cc:1034] running on GCE node
I20260811 00:24:56.919206 22423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:56.920961 22423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:56.922243 22423 hybrid_clock.cc:648] HybridClock initialized: now 1786407896922166 us; error 64 us; skew 500 ppm
I20260811 00:24:56.922652 22423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:56 dist-test-slave-f7th krb5kdc[22403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407896, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:56.926445 22423 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:56.928269 22423 webserver.cc:460] Webserver started at http://127.0.0.1:40573/ using document root <none> and password file <none>
I20260811 00:24:56.929108 22423 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:56.929244 22423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:56.929566 22423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:56.933002 22423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "cae0a0132110442e8c0909c925da4a38"
format_stamp: "Formatted at 2026-08-11 00:24:56 on dist-test-slave-f7th"
server_key: "aca9a689f4f17b25286a0589e74aa120"
server_key_iv: "69401467ea1e959a67198c95c9fc5e29"
server_key_version: "encryptionkey@0"
I20260811 00:24:56.933935 22423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "cae0a0132110442e8c0909c925da4a38"
format_stamp: "Formatted at 2026-08-11 00:24:56 on dist-test-slave-f7th"
server_key: "aca9a689f4f17b25286a0589e74aa120"
server_key_iv: "69401467ea1e959a67198c95c9fc5e29"
server_key_version: "encryptionkey@0"
I20260811 00:24:56.940167 22423 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:24:56.944502 22449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:56.946244 22423 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 00:24:56.946470 22423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "cae0a0132110442e8c0909c925da4a38"
format_stamp: "Formatted at 2026-08-11 00:24:56 on dist-test-slave-f7th"
server_key: "aca9a689f4f17b25286a0589e74aa120"
server_key_iv: "69401467ea1e959a67198c95c9fc5e29"
server_key_version: "encryptionkey@0"
I20260811 00:24:56.946709 22423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:56.953750 22423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:56.958817 22423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:56.959173 22423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:56.978488 22423 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34489
I20260811 00:24:56.978523 22542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34489 every 8 connection(s)
I20260811 00:24:56.980113 22423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:56.983517 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22423
I20260811 00:24:56.983732 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:56.984295 16670 external_mini_cluster.cc:1428] Setting key 86838ca3dedb510f02402fa3cd608b0a
I20260811 00:24:56.984920 22545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:56.993813 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:56.986398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:38953 (local address 127.0.0.1:34489)
0811 00:24:56.987262 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:56.987274 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:56.987319 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:24:56.989834 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:56.989864 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:56.990295 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:56.990675 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:56.991882 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:56.991892 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:56.991910 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:56.991926 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:56.992027 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:56.992051 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:56.992249 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:56.992362 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 00:24:56.992585 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":376,"threads_started":1}
I20260811 00:24:56.998502 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:56.999827 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38: Bootstrap starting.
I20260811 00:24:57.004573 22545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:57.006131 22545 log.cc:824] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38: Log is configured to *not* fsync() on all Append() calls
I20260811 00:24:57.008162 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:57.004835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:56669 (local address 127.0.0.1:34489)
0811 00:24:57.005049 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:57.005060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:57.005472 (+ 412us) server_negotiation.cc:407] Connection header received
0811 00:24:57.005660 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:57.005665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:57.005746 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:57.005890 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:57.007172 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:57.007176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:57.007180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:24:57.007198 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:57.007245 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:24:57.007263 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:57.007398 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:57.007623 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 00:24:57.007883 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:24:57.009848 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38: No bootstrap required, opened a new log
I20260811 00:24:57.017957 22545 raft_consensus.cc:348] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cae0a0132110442e8c0909c925da4a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260811 00:24:57.018435 22545 raft_consensus.cc:374] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:57.018560 22545 raft_consensus.cc:729] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae0a0132110442e8c0909c925da4a38, State: Initialized, Role: FOLLOWER
I20260811 00:24:57.019362 22545 consensus_queue.cc:260] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cae0a0132110442e8c0909c925da4a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260811 00:24:57.019665 22545 raft_consensus.cc:388] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:57.019838 22545 raft_consensus.cc:482] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:57.020030 22545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:57.022526 22545 raft_consensus.cc:504] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cae0a0132110442e8c0909c925da4a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260811 00:24:57.023130 22545 leader_election.cc:302] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cae0a0132110442e8c0909c925da4a38; no voters:
I20260811 00:24:57.023908 22545 leader_election.cc:288] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:57.024291 22559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:57.025128 22559 raft_consensus.cc:686] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [term 1 LEADER]: Becoming Leader. State: Replica: cae0a0132110442e8c0909c925da4a38, State: Running, Role: LEADER
I20260811 00:24:57.025862 22559 consensus_queue.cc:237] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cae0a0132110442e8c0909c925da4a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260811 00:24:57.026486 22545 sys_catalog.cc:564] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:57.029649 22560 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cae0a0132110442e8c0909c925da4a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cae0a0132110442e8c0909c925da4a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } } }
I20260811 00:24:57.030241 22560 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:57.030767 22565 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cae0a0132110442e8c0909c925da4a38. Latest consensus state: current_term: 1 leader_uuid: "cae0a0132110442e8c0909c925da4a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cae0a0132110442e8c0909c925da4a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } } }
I20260811 00:24:57.031100 22565 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:57.031255 22574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:57.040521 22574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:57.052868 22574 catalog_manager.cc:1347] Generated new cluster ID: c80e5a265c0444f4a5b4824f75678f2f
I20260811 00:24:57.053056 22574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:57.065165 22574 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:57.065687 22574 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:57.072916 22574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cae0a0132110442e8c0909c925da4a38: Generated new TSK 0
I20260811 00:24:57.074019 22574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:57.091033 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22423
W20260811 00:24:57.092424 22533 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:38953
W20260811 00:24:57.093364 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22423: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:24:57.093565 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22423
2026-08-11T00:24:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:57 dist-test-slave-f7th krb5kdc[22594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:57 dist-test-slave-f7th krb5kdc[22594](info): set up 2 sockets
Aug 11 00:24:57 dist-test-slave-f7th krb5kdc[22594](info): commencing operation
krb5kdc: starting...
W20260811 00:24:58.421846 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:24:58 dist-test-slave-f7th krb5kdc[22594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:24:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:24:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:24:58.780117 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:24:59.166667 22617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:24:59.167021 22617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:24:59.167128 22617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:24:59.174491 22617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:24:59.174621 22617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:24:59.174676 22617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:24:59.174737 22617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:24:59.174782 22617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:24:59.185231 22617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:24:59.186650 22617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:24:59.188097 22617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:24:59.200367 22630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:59.200469 22627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:24:59.200443 22628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:24:59.201387 22617 server_base.cc:1034] running on GCE node
I20260811 00:24:59.202339 22617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:24:59.203951 22617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:24:59.205171 22617 hybrid_clock.cc:648] HybridClock initialized: now 1786407899205127 us; error 42 us; skew 500 ppm
I20260811 00:24:59.205541 22617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:24:59 dist-test-slave-f7th krb5kdc[22594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407899, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:24:59.208957 22617 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:24:59.210700 22617 webserver.cc:460] Webserver started at http://127.0.0.1:35751/ using document root <none> and password file <none>
I20260811 00:24:59.211449 22617 fs_manager.cc:359] Metadata directory not provided
I20260811 00:24:59.211582 22617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:24:59.211917 22617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:24:59.215205 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "df5e984e28ad4b6085f0a9617667e840"
format_stamp: "Formatted at 2026-08-11 00:24:59 on dist-test-slave-f7th"
server_key: "1f1a6071f3555287a377451a34f10666"
server_key_iv: "66639e3c2dc9995b99364d6042e03179"
server_key_version: "encryptionkey@0"
I20260811 00:24:59.216063 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "df5e984e28ad4b6085f0a9617667e840"
format_stamp: "Formatted at 2026-08-11 00:24:59 on dist-test-slave-f7th"
server_key: "1f1a6071f3555287a377451a34f10666"
server_key_iv: "66639e3c2dc9995b99364d6042e03179"
server_key_version: "encryptionkey@0"
I20260811 00:24:59.222101 22617 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:24:59.226136 22644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:24:59.227720 22617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:24:59.227957 22617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "df5e984e28ad4b6085f0a9617667e840"
format_stamp: "Formatted at 2026-08-11 00:24:59 on dist-test-slave-f7th"
server_key: "1f1a6071f3555287a377451a34f10666"
server_key_iv: "66639e3c2dc9995b99364d6042e03179"
server_key_version: "encryptionkey@0"
I20260811 00:24:59.228184 22617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:24:59.240671 22617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:24:59.245558 22617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:24:59.245883 22617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:24:59.268092 22617 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38475
I20260811 00:24:59.268129 22736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38475 every 8 connection(s)
I20260811 00:24:59.269694 22617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:24:59.273871 22737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:24:59.277256 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22617
I20260811 00:24:59.277436 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:24:59.277884 16670 external_mini_cluster.cc:1428] Setting key 35304a5bd97f78ad895d6f301edb2c4c
I20260811 00:24:59.287794 22737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840: Bootstrap starting.
I20260811 00:24:59.291994 22737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840: Neither blocks nor log segments found. Creating new log.
I20260811 00:24:59.293154 22737 log.cc:824] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:24:59 dist-test-slave-f7th krb5kdc[22594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:24:59.296764 22737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840: No bootstrap required, opened a new log
I20260811 00:24:59.304322 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:59.279914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:58193 (local address 127.0.0.1:38475)
0811 00:24:59.280767 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:59.280785 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:59.280851 (+ 66us) server_negotiation.cc:407] Connection header received
0811 00:24:59.283131 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:59.283167 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:59.283858 (+ 691us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:59.284369 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:59.285950 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:59.288094 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:59.289583 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:59.290725 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:59.290956 (+ 231us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:59.295758 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:59.295768 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:59.295785 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:59.295828 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:59.299275 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:59.300453 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:59.300461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:59.300472 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:59.300714 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:59.301287 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:59.301291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:59.301293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:59.301957 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:59.302225 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:59.302778 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 00:24:59.303123 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":329,"threads_started":1}
I20260811 00:24:59.306085 22737 raft_consensus.cc:348] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5e984e28ad4b6085f0a9617667e840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260811 00:24:59.306466 22737 raft_consensus.cc:374] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:24:59.306579 22737 raft_consensus.cc:729] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df5e984e28ad4b6085f0a9617667e840, State: Initialized, Role: FOLLOWER
I20260811 00:24:59.307221 22737 consensus_queue.cc:260] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5e984e28ad4b6085f0a9617667e840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260811 00:24:59.307513 22737 raft_consensus.cc:388] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:24:59.307658 22737 raft_consensus.cc:482] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:24:59.307878 22737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:24:59.308575 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:24:59.311237 22737 raft_consensus.cc:504] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5e984e28ad4b6085f0a9617667e840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260811 00:24:59.312005 22737 leader_election.cc:302] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df5e984e28ad4b6085f0a9617667e840; no voters:
I20260811 00:24:59.312798 22737 leader_election.cc:288] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:24:59.312987 22750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:24:59.315191 22750 raft_consensus.cc:686] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [term 1 LEADER]: Becoming Leader. State: Replica: df5e984e28ad4b6085f0a9617667e840, State: Running, Role: LEADER
I20260811 00:24:59.315855 22737 sys_catalog.cc:564] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:24:59.316002 22750 consensus_queue.cc:237] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5e984e28ad4b6085f0a9617667e840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260811 00:24:59.319356 22754 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df5e984e28ad4b6085f0a9617667e840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5e984e28ad4b6085f0a9617667e840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } } }
I20260811 00:24:59.319939 22754 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:59.319813 22753 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df5e984e28ad4b6085f0a9617667e840. Latest consensus state: current_term: 1 leader_uuid: "df5e984e28ad4b6085f0a9617667e840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5e984e28ad4b6085f0a9617667e840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } } }
I20260811 00:24:59.320151 22753 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840 [sys.catalog]: This master's current role is: LEADER
I20260811 00:24:59.323478 22767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:24:59.333756 22767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:24:59.338835 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:24:59.314645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:52981 (local address 127.0.0.1:38475)
0811 00:24:59.314860 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:24:59.314868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:24:59.316109 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 00:24:59.316341 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:24:59.316346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:24:59.316440 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:24:59.316572 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:24:59.318906 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:59.320371 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:59.322578 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:24:59.323374 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:24:59.323510 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:24:59.328065 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:24:59.328073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:24:59.328079 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:24:59.328142 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:24:59.333016 (+ 4874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:59.334621 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:59.334628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:59.334633 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:59.334865 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:24:59.335569 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:24:59.335575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:24:59.335579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:24:59.336089 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:24:59.337906 (+ 1817us) server_negotiation.cc:1035] Waiting for connection context
0811 00:24:59.338117 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 00:24:59.338359 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":72}
I20260811 00:24:59.347725 22767 catalog_manager.cc:1347] Generated new cluster ID: fbcd05f6562a4b81b1ddf95f420d761b
I20260811 00:24:59.347962 22767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:24:59.377287 22767 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:24:59.379423 22767 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:24:59.395560 22767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df5e984e28ad4b6085f0a9617667e840: Generated new TSK 0
I20260811 00:24:59.396508 22767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:24:59.405097 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22617
I20260811 00:24:59.406486 22727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:24:59.502321 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22617
W20260811 00:24:59.530097 22592 connection.cc:535] client connection to 127.0.0.1:38475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38475): Connection reset by peer (error 104)
2026-08-11T00:24:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:24:59 dist-test-slave-f7th krb5kdc[22789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:24:59 dist-test-slave-f7th krb5kdc[22789](info): set up 2 sockets
Aug 11 00:24:59 dist-test-slave-f7th krb5kdc[22789](info): commencing operation
krb5kdc: starting...
W20260811 00:25:00.819087 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:01 dist-test-slave-f7th krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:25:01.173250 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45943
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:01.553187 22808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:01.553587 22808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:01.553735 22808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:01.561491 22808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:01.561650 22808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:01.561730 22808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:01.561817 22808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:01.561883 22808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:01.572506 22808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45943
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:01.573995 22808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:01.575425 22808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:01.589118 22826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:01.588989 22831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:01.589063 22828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:01.589496 22808 server_base.cc:1034] running on GCE node
I20260811 00:25:01.590423 22808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:01.592027 22808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:01.593292 22808 hybrid_clock.cc:648] HybridClock initialized: now 1786407901593212 us; error 68 us; skew 500 ppm
I20260811 00:25:01.593740 22808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:01 dist-test-slave-f7th krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407901, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:01.602754 22808 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:25:01.604681 22808 webserver.cc:460] Webserver started at http://127.0.0.1:40639/ using document root <none> and password file <none>
I20260811 00:25:01.605460 22808 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:01.605610 22808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:01.605924 22808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:01.609589 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "2b96ac4d93644d3ab10f226b9a17aa4f"
format_stamp: "Formatted at 2026-08-11 00:25:01 on dist-test-slave-f7th"
server_key: "62d0e1bdd56b6569d3fa3dda9ee31300"
server_key_iv: "86a68edac91bf00664da4643b9a2b661"
server_key_version: "encryptionkey@0"
I20260811 00:25:01.610476 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "2b96ac4d93644d3ab10f226b9a17aa4f"
format_stamp: "Formatted at 2026-08-11 00:25:01 on dist-test-slave-f7th"
server_key: "62d0e1bdd56b6569d3fa3dda9ee31300"
server_key_iv: "86a68edac91bf00664da4643b9a2b661"
server_key_version: "encryptionkey@0"
I20260811 00:25:01.616397 22808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:25:01.620647 22840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:01.622408 22808 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:01.622650 22808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "2b96ac4d93644d3ab10f226b9a17aa4f"
format_stamp: "Formatted at 2026-08-11 00:25:01 on dist-test-slave-f7th"
server_key: "62d0e1bdd56b6569d3fa3dda9ee31300"
server_key_iv: "86a68edac91bf00664da4643b9a2b661"
server_key_version: "encryptionkey@0"
I20260811 00:25:01.622891 22808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:01.659981 22808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:01.664986 22808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:01.665297 22808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:01.685277 22808 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45943
I20260811 00:25:01.685308 22941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45943 every 8 connection(s)
I20260811 00:25:01.686920 22808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:01.689462 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 22808
I20260811 00:25:01.689663 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:01.690124 16670 external_mini_cluster.cc:1428] Setting key 48facb97ff414f43f9d017f0b4c9392a
I20260811 00:25:01.691843 22942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:01.706106 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:01.692194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:42865 (local address 127.0.0.1:45943)
0811 00:25:01.693069 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:01.693081 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:01.693127 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:25:01.694839 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:01.694866 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:01.695335 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:01.695734 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:01.696443 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:01.697899 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:01.700001 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:01.701151 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:01.701340 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:01.703291 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:01.703302 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:01.703320 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:01.703993 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:01.704120 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:01.704164 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:01.704351 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:01.704801 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 00:25:01.704995 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":366,"threads_started":1}
I20260811 00:25:01.706665 22942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f: Bootstrap starting.
I20260811 00:25:01.710345 22942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:01.710583 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:01.711524 22942 log.cc:824] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:01.714941 22942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f: No bootstrap required, opened a new log
I20260811 00:25:01.724117 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:01.715107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:34701 (local address 127.0.0.1:45943)
0811 00:25:01.715296 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:01.715303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:01.717314 (+ 2011us) server_negotiation.cc:407] Connection header received
0811 00:25:01.717497 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:01.717506 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:01.717600 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:01.717742 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:01.718387 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:01.719572 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:01.721177 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:01.721874 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:01.722023 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:01.723199 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:01.723206 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:01.723211 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:01.723238 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:01.723307 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:01.723335 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:01.723459 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:01.723710 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 00:25:01.723776 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:25:01.724761 22942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45943 } }
I20260811 00:25:01.725220 22942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:01.725337 22942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b96ac4d93644d3ab10f226b9a17aa4f, State: Initialized, Role: FOLLOWER
I20260811 00:25:01.726027 22942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45943 } }
I20260811 00:25:01.726267 22942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:01.726406 22942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:01.726603 22942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:01.729691 22942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45943 } }
I20260811 00:25:01.730299 22942 leader_election.cc:302] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b96ac4d93644d3ab10f226b9a17aa4f; no voters:
I20260811 00:25:01.730935 22942 leader_election.cc:288] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:01.731097 22956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:01.731784 22956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [term 1 LEADER]: Becoming Leader. State: Replica: 2b96ac4d93644d3ab10f226b9a17aa4f, State: Running, Role: LEADER
I20260811 00:25:01.732508 22956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45943 } }
I20260811 00:25:01.734575 22942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:01.735610 22957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b96ac4d93644d3ab10f226b9a17aa4f. Latest consensus state: current_term: 1 leader_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45943 } } }
I20260811 00:25:01.736016 22957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:01.737126 22958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b96ac4d93644d3ab10f226b9a17aa4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45943 } } }
I20260811 00:25:01.737463 22958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:01.738301 22962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:01.743738 22962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:01.754690 22962 catalog_manager.cc:1347] Generated new cluster ID: 8f1027df610741d995edae8a6bbbf602
I20260811 00:25:01.754828 22962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:01.793305 22962 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:01.794862 22962 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:01.804991 22962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b96ac4d93644d3ab10f226b9a17aa4f: Generated new TSK 0
I20260811 00:25:01.806108 22962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:01.824944 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22808
W20260811 00:25:01.826432 22934 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:42865
W20260811 00:25:01.828723 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 22808: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:01.828920 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 22808
W20260811 00:25:01.876909 22785 connection.cc:535] client connection to 127.0.0.1:45943 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45943): Connection reset by peer (error 104)
2026-08-11T00:25:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:01 dist-test-slave-f7th krb5kdc[22991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:01 dist-test-slave-f7th krb5kdc[22991](info): set up 2 sockets
Aug 11 00:25:01 dist-test-slave-f7th krb5kdc[22991](info): commencing operation
krb5kdc: starting...
W20260811 00:25:03.191462 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:03 dist-test-slave-f7th krb5kdc[22991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:25:03.552883 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:03.954236 23014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:03.954627 23014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:03.954767 23014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:03.962725 23014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:03.963019 23014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:03.963189 23014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:03.963295 23014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:03.963363 23014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:03.974946 23014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:03.976485 23014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:03.977974 23014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:03.992326 23022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:03.992398 23021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:03.992354 23028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:03.993287 23014 server_base.cc:1034] running on GCE node
I20260811 00:25:03.994246 23014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:03.996243 23014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:03.999699 23014 hybrid_clock.cc:648] HybridClock initialized: now 1786407903999591 us; error 121 us; skew 500 ppm
I20260811 00:25:04.000212 23014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:03 dist-test-slave-f7th krb5kdc[22991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407903, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:04.004004 23014 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:25:04.006124 23014 webserver.cc:460] Webserver started at http://127.0.0.1:35901/ using document root <none> and password file <none>
I20260811 00:25:04.006956 23014 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:04.007100 23014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:04.007409 23014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:04.011132 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "6003a03150784d6fa01167c121e7c641"
format_stamp: "Formatted at 2026-08-11 00:25:04 on dist-test-slave-f7th"
server_key: "26aa0bf801cad24c94d6be9b7334dd71"
server_key_iv: "9b8a88abd2a4ec3558be9c0aaeb6d06e"
server_key_version: "encryptionkey@0"
I20260811 00:25:04.012068 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "6003a03150784d6fa01167c121e7c641"
format_stamp: "Formatted at 2026-08-11 00:25:04 on dist-test-slave-f7th"
server_key: "26aa0bf801cad24c94d6be9b7334dd71"
server_key_iv: "9b8a88abd2a4ec3558be9c0aaeb6d06e"
server_key_version: "encryptionkey@0"
I20260811 00:25:04.019007 23014 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260811 00:25:04.023626 23036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:04.025413 23014 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:04.025651 23014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "6003a03150784d6fa01167c121e7c641"
format_stamp: "Formatted at 2026-08-11 00:25:04 on dist-test-slave-f7th"
server_key: "26aa0bf801cad24c94d6be9b7334dd71"
server_key_iv: "9b8a88abd2a4ec3558be9c0aaeb6d06e"
server_key_version: "encryptionkey@0"
I20260811 00:25:04.025895 23014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:04.032953 23014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:04.038491 23014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:04.038844 23014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:04.058941 23014 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43807
I20260811 00:25:04.058964 23123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43807 every 8 connection(s)
I20260811 00:25:04.060487 23014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:04.061043 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23014
I20260811 00:25:04.061296 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:04.061908 16670 external_mini_cluster.cc:1428] Setting key 0c8021d22be0f866befc94b1591ef75b
I20260811 00:25:04.066197 23124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:04.071820 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:04.064351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:36667 (local address 127.0.0.1:43807)
0811 00:25:04.065244 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:04.065258 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:04.065312 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:25:04.067657 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:04.067689 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:04.068157 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:04.068687 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:04.069697 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:04.069706 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:04.069727 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:04.069749 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:04.069855 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:04.069900 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:04.070138 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:04.070279 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 00:25:04.070487 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":353,"threads_started":1}
I20260811 00:25:04.076520 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:04.080204 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641: Bootstrap starting.
I20260811 00:25:04.084214 23124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:04.084216 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:04.081533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:36715 (local address 127.0.0.1:43807)
0811 00:25:04.081759 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:04.081768 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:04.082261 (+ 493us) server_negotiation.cc:407] Connection header received
0811 00:25:04.082433 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:04.082437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:04.082498 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:04.082595 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:04.083551 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:04.083554 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:04.083557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:04.083571 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:04.083608 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:04.083622 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:04.083704 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:04.083831 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 00:25:04.083883 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:25:04.085683 23124 log.cc:824] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:04.089112 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641: No bootstrap required, opened a new log
I20260811 00:25:04.097215 23124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6003a03150784d6fa01167c121e7c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 00:25:04.097640 23124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:04.097719 23124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6003a03150784d6fa01167c121e7c641, State: Initialized, Role: FOLLOWER
I20260811 00:25:04.098292 23124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6003a03150784d6fa01167c121e7c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 00:25:04.098522 23124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:04.098654 23124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:04.098803 23124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:04.101599 23124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6003a03150784d6fa01167c121e7c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 00:25:04.102139 23124 leader_election.cc:302] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6003a03150784d6fa01167c121e7c641; no voters:
I20260811 00:25:04.102763 23124 leader_election.cc:288] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:04.102921 23137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:04.103621 23137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [term 1 LEADER]: Becoming Leader. State: Replica: 6003a03150784d6fa01167c121e7c641, State: Running, Role: LEADER
I20260811 00:25:04.104218 23137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6003a03150784d6fa01167c121e7c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 00:25:04.104955 23124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:04.107256 23139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6003a03150784d6fa01167c121e7c641. Latest consensus state: current_term: 1 leader_uuid: "6003a03150784d6fa01167c121e7c641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6003a03150784d6fa01167c121e7c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } } }
I20260811 00:25:04.108492 23139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:04.108067 23138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6003a03150784d6fa01167c121e7c641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6003a03150784d6fa01167c121e7c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } } }
I20260811 00:25:04.109908 23138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:04.110945 23149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:04.116814 23149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:04.127978 23149 catalog_manager.cc:1347] Generated new cluster ID: 84f5628610e048d09bf2eeba610fed23
I20260811 00:25:04.128126 23149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:04.145416 23149 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:04.146059 23149 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:04.153723 23149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6003a03150784d6fa01167c121e7c641: Generated new TSK 0
I20260811 00:25:04.154736 23149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:04.160918 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23014
W20260811 00:25:04.162478 23119 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:36667
W20260811 00:25:04.163372 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23014: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:04.163530 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23014
2026-08-11T00:25:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:04 dist-test-slave-f7th krb5kdc[23174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:04 dist-test-slave-f7th krb5kdc[23174](info): set up 2 sockets
Aug 11 00:25:04 dist-test-slave-f7th krb5kdc[23174](info): commencing operation
krb5kdc: starting...
W20260811 00:25:05.529023 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:05 dist-test-slave-f7th krb5kdc[23174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:05.900347 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36987
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:36987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:06.277236 23204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:06.277583 23204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:06.277719 23204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:06.285346 23204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:06.285492 23204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:06.285543 23204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:06.285630 23204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:06.285681 23204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:06.296380 23204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:36987
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:36987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:06.297884 23204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:06.299319 23204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:06.312265 23213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:06.312311 23215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:06.312225 23212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:06.313335 23204 server_base.cc:1034] running on GCE node
I20260811 00:25:06.314314 23204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:06.316179 23204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:06.317451 23204 hybrid_clock.cc:648] HybridClock initialized: now 1786407906317403 us; error 43 us; skew 500 ppm
I20260811 00:25:06.317903 23204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:06 dist-test-slave-f7th krb5kdc[23174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:06.321833 23204 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:06.323753 23204 webserver.cc:460] Webserver started at http://127.16.71.190:38415/ using document root <none> and password file <none>
I20260811 00:25:06.324566 23204 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:06.324760 23204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:06.325110 23204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:06.329370 23204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "94d2f6dea54a4e4b98492bf4c43a0f87"
format_stamp: "Formatted at 2026-08-11 00:25:06 on dist-test-slave-f7th"
server_key: "e420df457ab5e1b55fa042826e8815e2"
server_key_iv: "07c277aaad69cee9a5ebde0dc8e15960"
server_key_version: "encryptionkey@0"
I20260811 00:25:06.330406 23204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "94d2f6dea54a4e4b98492bf4c43a0f87"
format_stamp: "Formatted at 2026-08-11 00:25:06 on dist-test-slave-f7th"
server_key: "e420df457ab5e1b55fa042826e8815e2"
server_key_iv: "07c277aaad69cee9a5ebde0dc8e15960"
server_key_version: "encryptionkey@0"
I20260811 00:25:06.336961 23204 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:25:06.341572 23226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:06.343565 23204 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:25:06.343777 23204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "94d2f6dea54a4e4b98492bf4c43a0f87"
format_stamp: "Formatted at 2026-08-11 00:25:06 on dist-test-slave-f7th"
server_key: "e420df457ab5e1b55fa042826e8815e2"
server_key_iv: "07c277aaad69cee9a5ebde0dc8e15960"
server_key_version: "encryptionkey@0"
I20260811 00:25:06.344059 23204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:06.361136 23204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:06.366545 23204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:06.366866 23204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:06.389295 23204 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:36987
I20260811 00:25:06.389326 23303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:36987 every 8 connection(s)
I20260811 00:25:06.390900 23204 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:06.395385 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:06.399132 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23204
I20260811 00:25:06.399343 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:06.399895 16670 external_mini_cluster.cc:1428] Setting key ce0af56f509fcb9f758a68a844a23fc8
I20260811 00:25:06.409734 23304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87: Bootstrap starting.
I20260811 00:25:06.414108 23304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:06.415818 23304 log.cc:824] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:25:06 dist-test-slave-f7th krb5kdc[23174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:25:06.419816 23304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87: No bootstrap required, opened a new log
I20260811 00:25:06.427629 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:06.402198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50395 (local address 127.16.71.190:36987)
0811 00:25:06.403069 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:06.403083 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:06.403131 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:25:06.405015 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:06.405045 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:06.405558 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:06.405949 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:06.408145 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:06.409772 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:06.411838 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:06.412725 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:06.412902 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:06.418781 (+ 5879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:06.418791 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:06.418809 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:06.418853 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:06.422276 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:06.423468 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:06.423476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:06.423484 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:06.423689 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:06.424331 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:06.424335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:06.424337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:06.425103 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:06.425386 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:06.425985 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 00:25:06.426402 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":383,"threads_started":1}
I20260811 00:25:06.429419 23304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36987 } }
I20260811 00:25:06.429832 23304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:06.429950 23304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d2f6dea54a4e4b98492bf4c43a0f87, State: Initialized, Role: FOLLOWER
I20260811 00:25:06.430598 23304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36987 } }
I20260811 00:25:06.430945 23304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:06.431109 23304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:06.431289 23304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:06.432354 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:06.434495 23304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36987 } }
I20260811 00:25:06.435218 23304 leader_election.cc:302] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94d2f6dea54a4e4b98492bf4c43a0f87; no voters:
I20260811 00:25:06.436007 23304 leader_election.cc:288] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:06.436149 23313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:06.437541 23313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [term 1 LEADER]: Becoming Leader. State: Replica: 94d2f6dea54a4e4b98492bf4c43a0f87, State: Running, Role: LEADER
I20260811 00:25:06.438287 23313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36987 } }
I20260811 00:25:06.438849 23304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:06.441814 23318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d2f6dea54a4e4b98492bf4c43a0f87. Latest consensus state: current_term: 1 leader_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36987 } } }
I20260811 00:25:06.442304 23318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:06.443225 23316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d2f6dea54a4e4b98492bf4c43a0f87" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 36987 } } }
I20260811 00:25:06.443610 23316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:06.446327 23324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:06.453744 23324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:06.465206 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:06.439917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54067 (local address 127.16.71.190:36987)
0811 00:25:06.440101 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:06.440107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:06.440132 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:25:06.440330 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:06.440337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:06.440439 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:06.440576 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:06.443285 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:06.444585 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:06.446227 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:06.446978 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:06.447134 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:06.451180 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:06.451190 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:06.451196 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:06.451263 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:06.456714 (+ 5451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:06.458212 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:06.458220 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:06.458224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:06.458449 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:06.459251 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:06.459257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:06.459261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:06.459778 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:06.460128 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:06.462565 (+ 2437us) server_negotiation.cc:299] Negotiation successful
0811 00:25:06.462847 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:25:06.470367 23324 catalog_manager.cc:1347] Generated new cluster ID: 609b207e5bba4080917f084088fb6e0a
I20260811 00:25:06.470528 23324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:06.491151 23324 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:06.493319 23324 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:06.501857 23324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94d2f6dea54a4e4b98492bf4c43a0f87: Generated new TSK 0
I20260811 00:25:06.502785 23324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:06.517428 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23204
I20260811 00:25:06.518945 23296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:06.615620 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23204
W20260811 00:25:06.649338 23172 connection.cc:535] client connection to 127.16.71.190:36987 recv error: Network error: failed to read from TLS socket (remote: 127.16.71.190:36987): Connection reset by peer (error 104)
2026-08-11T00:25:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:06 dist-test-slave-f7th krb5kdc[23348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:06 dist-test-slave-f7th krb5kdc[23348](info): set up 2 sockets
Aug 11 00:25:06 dist-test-slave-f7th krb5kdc[23348](info): commencing operation
krb5kdc: starting...
W20260811 00:25:07.974239 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:08 dist-test-slave-f7th krb5kdc[23348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:08.336421 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43531
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:43531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:08.716470 23369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:08.716917 23369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:08.717094 23369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:08.725049 23369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:08.725185 23369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:08.725265 23369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:08.725359 23369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:08.725428 23369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:08.737457 23369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:43531
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:08.738919 23369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:08.740401 23369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:08.752897 23384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:08.752882 23379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:08.752938 23381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:08.753824 23369 server_base.cc:1034] running on GCE node
I20260811 00:25:08.754717 23369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:08.756222 23369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:08.757479 23369 hybrid_clock.cc:648] HybridClock initialized: now 1786407908757432 us; error 30 us; skew 500 ppm
I20260811 00:25:08.757869 23369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:08 dist-test-slave-f7th krb5kdc[23348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407908, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:08.761322 23369 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:08.763052 23369 webserver.cc:460] Webserver started at http://127.16.71.190:34993/ using document root <none> and password file <none>
I20260811 00:25:08.763769 23369 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:08.763891 23369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:08.764176 23369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:08.767688 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "01a2e042f43e4ae78a8c0083298a24a9"
format_stamp: "Formatted at 2026-08-11 00:25:08 on dist-test-slave-f7th"
server_key: "3bf3231777e090cbb576c50e3b9b6949"
server_key_iv: "3d16933bfdcb286ed0d8c4808ff833c1"
server_key_version: "encryptionkey@0"
I20260811 00:25:08.768667 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "01a2e042f43e4ae78a8c0083298a24a9"
format_stamp: "Formatted at 2026-08-11 00:25:08 on dist-test-slave-f7th"
server_key: "3bf3231777e090cbb576c50e3b9b6949"
server_key_iv: "3d16933bfdcb286ed0d8c4808ff833c1"
server_key_version: "encryptionkey@0"
I20260811 00:25:08.774705 23369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:25:08.778857 23398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:08.780584 23369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:25:08.780881 23369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "01a2e042f43e4ae78a8c0083298a24a9"
format_stamp: "Formatted at 2026-08-11 00:25:08 on dist-test-slave-f7th"
server_key: "3bf3231777e090cbb576c50e3b9b6949"
server_key_iv: "3d16933bfdcb286ed0d8c4808ff833c1"
server_key_version: "encryptionkey@0"
I20260811 00:25:08.781116 23369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:08.808359 23369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:08.813203 23369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:08.813513 23369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:08.833698 23369 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:43531
I20260811 00:25:08.833712 23481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:43531 every 8 connection(s)
I20260811 00:25:08.835295 23369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:08.839692 23484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:08.843089 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23369
I20260811 00:25:08.843295 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:08.843770 16670 external_mini_cluster.cc:1428] Setting key 11d9093d5dcabae19f5cef2411b14363
I20260811 00:25:08.852715 23484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9: Bootstrap starting.
I20260811 00:25:08.856410 23484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:08.857785 23484 log.cc:824] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:08.858451 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:08.845876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34245 (local address 127.16.71.190:43531)
0811 00:25:08.846763 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:08.846776 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:08.846838 (+ 62us) server_negotiation.cc:407] Connection header received
0811 00:25:08.848767 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:08.848799 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:08.849309 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:08.849761 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:08.850428 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:08.851903 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:08.853487 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:08.854241 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:08.854458 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:08.855853 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:08.855861 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:08.855879 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:08.856476 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:08.856581 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:08.856648 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:08.856835 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:08.857130 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 00:25:08.857323 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":337,"threads_started":1}
I20260811 00:25:08.861184 23484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9: No bootstrap required, opened a new log
I20260811 00:25:08.862741 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:08.870059 23484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2e042f43e4ae78a8c0083298a24a9" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43531 } }
I20260811 00:25:08.870620 23484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:08.870748 23484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a2e042f43e4ae78a8c0083298a24a9, State: Initialized, Role: FOLLOWER
I20260811 00:25:08.871397 23484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2e042f43e4ae78a8c0083298a24a9" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43531 } }
I20260811 00:25:08.871652 23484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:08.871793 23484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:08.871969 23484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:08.875623 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:08.867390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36647 (local address 127.16.71.190:43531)
0811 00:25:08.867614 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:08.867621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:08.868072 (+ 451us) server_negotiation.cc:407] Connection header received
0811 00:25:08.868236 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:08.868242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:08.868346 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:08.868481 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:08.869268 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:08.870348 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:08.872082 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:08.873062 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:08.873230 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:08.874701 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:08.874708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:08.874713 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:08.874739 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:08.874821 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:08.874850 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:08.874981 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:08.875202 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 00:25:08.875268 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:25:08.876089 23484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2e042f43e4ae78a8c0083298a24a9" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43531 } }
I20260811 00:25:08.876873 23484 leader_election.cc:302] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01a2e042f43e4ae78a8c0083298a24a9; no voters:
I20260811 00:25:08.877650 23484 leader_election.cc:288] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:08.877729 23498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:08.878477 23498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [term 1 LEADER]: Becoming Leader. State: Replica: 01a2e042f43e4ae78a8c0083298a24a9, State: Running, Role: LEADER
I20260811 00:25:08.879130 23498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2e042f43e4ae78a8c0083298a24a9" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43531 } }
I20260811 00:25:08.879881 23484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:08.882019 23499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01a2e042f43e4ae78a8c0083298a24a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2e042f43e4ae78a8c0083298a24a9" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43531 } } }
I20260811 00:25:08.882481 23499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:08.884851 23501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01a2e042f43e4ae78a8c0083298a24a9. Latest consensus state: current_term: 1 leader_uuid: "01a2e042f43e4ae78a8c0083298a24a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2e042f43e4ae78a8c0083298a24a9" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43531 } } }
I20260811 00:25:08.885228 23501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:08.885942 23505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:08.892098 23505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:08.905867 23505 catalog_manager.cc:1347] Generated new cluster ID: 28701ec06eb84d609efb0e5521235b0b
I20260811 00:25:08.906030 23505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:08.923975 23505 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:08.925794 23505 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:08.944446 23505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01a2e042f43e4ae78a8c0083298a24a9: Generated new TSK 0
I20260811 00:25:08.945611 23505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:08.961910 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23369
W20260811 00:25:08.963627 23476 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34245
W20260811 00:25:08.964566 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:08.964759 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23369
W20260811 00:25:08.993366 23346 connection.cc:535] client connection to 127.16.71.190:43531 recv error: Network error: failed to read from TLS socket (remote: 127.16.71.190:43531): Connection reset by peer (error 104)
2026-08-11T00:25:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:09 dist-test-slave-f7th krb5kdc[23529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:09 dist-test-slave-f7th krb5kdc[23529](info): set up 2 sockets
Aug 11 00:25:09 dist-test-slave-f7th krb5kdc[23529](info): commencing operation
krb5kdc: starting...
W20260811 00:25:10.297597 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.006s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:10 dist-test-slave-f7th krb5kdc[23529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:10.658166 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:33383
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:33383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:11.034497 23557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:11.034847 23557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:11.034996 23557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:11.042580 23557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:11.042727 23557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:11.042819 23557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:11.042902 23557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:11.042972 23557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:11.053391 23557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:33383
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:33383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:11.054859 23557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:11.056315 23557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:11.069777 23566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:11.069864 23565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:11.069804 23568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:11.071012 23557 server_base.cc:1034] running on GCE node
I20260811 00:25:11.072045 23557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:11.073777 23557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:11.075071 23557 hybrid_clock.cc:648] HybridClock initialized: now 1786407911074972 us; error 56 us; skew 500 ppm
I20260811 00:25:11.075492 23557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:11 dist-test-slave-f7th krb5kdc[23529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407911, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:11.078953 23557 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:11.080719 23557 webserver.cc:460] Webserver started at http://127.16.71.190:38013/ using document root <none> and password file <none>
I20260811 00:25:11.081511 23557 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:11.081655 23557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:11.081990 23557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:11.085407 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "f90098ab50ec4f558884f4a88c6dfcb5"
format_stamp: "Formatted at 2026-08-11 00:25:11 on dist-test-slave-f7th"
server_key: "3e1e2260e6c10cf21c787b359486cac9"
server_key_iv: "6e943e429c6c931d968b9f9b60e90cce"
server_key_version: "encryptionkey@0"
I20260811 00:25:11.086308 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "f90098ab50ec4f558884f4a88c6dfcb5"
format_stamp: "Formatted at 2026-08-11 00:25:11 on dist-test-slave-f7th"
server_key: "3e1e2260e6c10cf21c787b359486cac9"
server_key_iv: "6e943e429c6c931d968b9f9b60e90cce"
server_key_version: "encryptionkey@0"
I20260811 00:25:11.092592 23557 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:25:11.096832 23578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:11.098453 23557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:25:11.098686 23557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "f90098ab50ec4f558884f4a88c6dfcb5"
format_stamp: "Formatted at 2026-08-11 00:25:11 on dist-test-slave-f7th"
server_key: "3e1e2260e6c10cf21c787b359486cac9"
server_key_iv: "6e943e429c6c931d968b9f9b60e90cce"
server_key_version: "encryptionkey@0"
I20260811 00:25:11.098917 23557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:11.105237 23557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:11.109754 23557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:11.110060 23557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:11.130741 23557 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:33383
I20260811 00:25:11.130780 23666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:33383 every 8 connection(s)
I20260811 00:25:11.132290 23557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:11.134428 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23557
I20260811 00:25:11.134596 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:11.135036 16670 external_mini_cluster.cc:1428] Setting key 1434084acceb26d83652511fbeace0e3
I20260811 00:25:11.137061 23667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:11.144598 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:11.136907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58627 (local address 127.16.71.190:33383)
0811 00:25:11.138030 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:11.138047 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:11.138102 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:25:11.140326 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:11.140360 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:11.140920 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:11.141422 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:11.142472 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:11.142482 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:11.142506 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:11.142531 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:11.142622 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:11.142653 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:11.142908 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:11.143042 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 00:25:11.143268 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":593,"threads_started":1}
I20260811 00:25:11.148996 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:11.152040 23667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5: Bootstrap starting.
I20260811 00:25:11.156378 23667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:11.157416 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:11.154629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49251 (local address 127.16.71.190:33383)
0811 00:25:11.154849 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:11.154855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:11.155225 (+ 370us) server_negotiation.cc:407] Connection header received
0811 00:25:11.155405 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:11.155409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:11.155470 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:11.155557 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:11.156648 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:11.156655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:11.156659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:11.156682 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:11.156728 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:11.156747 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:11.156867 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:11.157047 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 00:25:11.157102 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:25:11.157899 23667 log.cc:824] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:11.161161 23667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5: No bootstrap required, opened a new log
I20260811 00:25:11.169464 23667 raft_consensus.cc:348] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33383 } }
I20260811 00:25:11.169983 23667 raft_consensus.cc:374] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:11.170104 23667 raft_consensus.cc:729] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90098ab50ec4f558884f4a88c6dfcb5, State: Initialized, Role: FOLLOWER
I20260811 00:25:11.170786 23667 consensus_queue.cc:260] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33383 } }
I20260811 00:25:11.171051 23667 raft_consensus.cc:388] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:11.171195 23667 raft_consensus.cc:482] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:11.171386 23667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:11.174000 23667 raft_consensus.cc:504] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33383 } }
I20260811 00:25:11.174650 23667 leader_election.cc:302] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f90098ab50ec4f558884f4a88c6dfcb5; no voters:
I20260811 00:25:11.175462 23667 leader_election.cc:288] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:11.175635 23682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:11.176589 23682 raft_consensus.cc:686] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [term 1 LEADER]: Becoming Leader. State: Replica: f90098ab50ec4f558884f4a88c6dfcb5, State: Running, Role: LEADER
I20260811 00:25:11.177249 23682 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33383 } }
I20260811 00:25:11.177788 23667 sys_catalog.cc:564] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:11.180105 23685 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f90098ab50ec4f558884f4a88c6dfcb5. Latest consensus state: current_term: 1 leader_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33383 } } }
I20260811 00:25:11.180651 23685 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:11.180547 23684 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90098ab50ec4f558884f4a88c6dfcb5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33383 } } }
I20260811 00:25:11.181107 23684 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:11.182799 23698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:11.188802 23698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:11.199270 23698 catalog_manager.cc:1347] Generated new cluster ID: 6037b00a285f403bba3fe51446bb1808
I20260811 00:25:11.199445 23698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:11.218710 23698 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:11.219322 23698 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:11.229679 23698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f90098ab50ec4f558884f4a88c6dfcb5: Generated new TSK 0
I20260811 00:25:11.230947 23698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:11.242074 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23557
W20260811 00:25:11.243428 23659 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58627
W20260811 00:25:11.244242 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23557: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:11.244428 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23557
2026-08-11T00:25:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:11 dist-test-slave-f7th krb5kdc[23719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:11 dist-test-slave-f7th krb5kdc[23719](info): set up 2 sockets
Aug 11 00:25:11 dist-test-slave-f7th krb5kdc[23719](info): commencing operation
krb5kdc: starting...
W20260811 00:25:12.599851 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:12 dist-test-slave-f7th krb5kdc[23719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:12.969636 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:33707
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:13.351401 23738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:13.351756 23738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:13.351900 23738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:13.359941 23738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:13.360132 23738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:13.360221 23738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:13.360314 23738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:13.360384 23738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:13.371227 23738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:33707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:13.372824 23738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:13.374287 23738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:13.387356 23746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:13.387473 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:13.387748 23749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:13.389216 23738 server_base.cc:1034] running on GCE node
I20260811 00:25:13.390439 23738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:13.392670 23738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:13.393967 23738 hybrid_clock.cc:648] HybridClock initialized: now 1786407913393909 us; error 97 us; skew 500 ppm
I20260811 00:25:13.394500 23738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:13 dist-test-slave-f7th krb5kdc[23719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407913, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:13.403409 23738 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:13.405691 23738 webserver.cc:460] Webserver started at http://127.16.71.190:45615/ using document root <none> and password file <none>
I20260811 00:25:13.406790 23738 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:13.407027 23738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:13.407469 23738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:13.412607 23738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "f4240bbfd9c043cbbb492542d68a3a81"
format_stamp: "Formatted at 2026-08-11 00:25:13 on dist-test-slave-f7th"
server_key: "eb81ac09b6a4fd127fd06335e8dc99d3"
server_key_iv: "b5fe993d793625a5ca2cc204cb6ecb40"
server_key_version: "encryptionkey@0"
I20260811 00:25:13.413944 23738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "f4240bbfd9c043cbbb492542d68a3a81"
format_stamp: "Formatted at 2026-08-11 00:25:13 on dist-test-slave-f7th"
server_key: "eb81ac09b6a4fd127fd06335e8dc99d3"
server_key_iv: "b5fe993d793625a5ca2cc204cb6ecb40"
server_key_version: "encryptionkey@0"
I20260811 00:25:13.420066 23738 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 00:25:13.424149 23762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:13.425772 23738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:25:13.425966 23738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "f4240bbfd9c043cbbb492542d68a3a81"
format_stamp: "Formatted at 2026-08-11 00:25:13 on dist-test-slave-f7th"
server_key: "eb81ac09b6a4fd127fd06335e8dc99d3"
server_key_iv: "b5fe993d793625a5ca2cc204cb6ecb40"
server_key_version: "encryptionkey@0"
I20260811 00:25:13.426187 23738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:13.440866 23738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:13.445840 23738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:13.446120 23738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:13.466799 23738 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:33707
I20260811 00:25:13.466850 23841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:33707 every 8 connection(s)
I20260811 00:25:13.468372 23738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:13.472138 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23738
I20260811 00:25:13.472322 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:13.472913 16670 external_mini_cluster.cc:1428] Setting key c1ab86239c8ed73855fa491fc2f6b3f9
I20260811 00:25:13.473335 23845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:13.489689 23845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81: Bootstrap starting.
Aug 11 00:25:13 dist-test-slave-f7th krb5kdc[23719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:25:13.493767 23845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:13.495281 23845 log.cc:824] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:13.499137 23845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81: No bootstrap required, opened a new log
I20260811 00:25:13.501464 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:13.474865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44731 (local address 127.16.71.190:33707)
0811 00:25:13.475723 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:13.475736 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:13.475780 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:25:13.478461 (+ 2681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:13.478495 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:13.479028 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:13.479453 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:13.481823 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:13.483290 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:13.485289 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:13.486193 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:13.486381 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:13.492202 (+ 5821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:13.492215 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:13.492235 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:13.492295 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:13.496329 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:13.497289 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:13.497299 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:13.497308 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:13.497565 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:13.497958 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:13.497964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:13.497967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:13.498737 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:13.499042 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:13.499514 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 00:25:13.500011 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":381,"threads_started":1}
I20260811 00:25:13.505872 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:13.508442 23845 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4240bbfd9c043cbbb492542d68a3a81" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33707 } }
I20260811 00:25:13.509017 23845 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:13.509123 23845 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4240bbfd9c043cbbb492542d68a3a81, State: Initialized, Role: FOLLOWER
I20260811 00:25:13.509768 23845 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4240bbfd9c043cbbb492542d68a3a81" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33707 } }
I20260811 00:25:13.510007 23845 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:13.510166 23845 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:13.510370 23845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:13.513777 23845 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4240bbfd9c043cbbb492542d68a3a81" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33707 } }
I20260811 00:25:13.514451 23845 leader_election.cc:302] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4240bbfd9c043cbbb492542d68a3a81; no voters:
I20260811 00:25:13.515197 23845 leader_election.cc:288] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:13.515316 23867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:13.516542 23867 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [term 1 LEADER]: Becoming Leader. State: Replica: f4240bbfd9c043cbbb492542d68a3a81, State: Running, Role: LEADER
I20260811 00:25:13.517357 23867 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4240bbfd9c043cbbb492542d68a3a81" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33707 } }
I20260811 00:25:13.517791 23845 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:13.522367 23868 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4240bbfd9c043cbbb492542d68a3a81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4240bbfd9c043cbbb492542d68a3a81" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33707 } } }
I20260811 00:25:13.522804 23868 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:13.530108 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:13.510371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41887 (local address 127.16.71.190:33707)
0811 00:25:13.510599 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:13.510607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:13.511026 (+ 419us) server_negotiation.cc:407] Connection header received
0811 00:25:13.511202 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:13.511206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:13.511284 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:13.511406 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:13.513340 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:13.514114 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:13.515983 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:13.516721 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:13.516892 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:13.520475 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:13.520484 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:13.520490 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:13.520547 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:13.524896 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:13.526110 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:13.526118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:13.526122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:13.526344 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:13.526939 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:13.526945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:13.526949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:13.527388 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:13.529060 (+ 1672us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:13.529277 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 00:25:13.529615 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:25:13.532794 23878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:13.523023 23872 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4240bbfd9c043cbbb492542d68a3a81. Latest consensus state: current_term: 1 leader_uuid: "f4240bbfd9c043cbbb492542d68a3a81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4240bbfd9c043cbbb492542d68a3a81" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 33707 } } }
I20260811 00:25:13.533064 23872 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:13.540645 23878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:13.552335 23878 catalog_manager.cc:1347] Generated new cluster ID: 80d90ba8b1534fd2b911f1333386ab96
I20260811 00:25:13.552474 23878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:13.569459 23878 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:13.571777 23878 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:13.583004 23878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4240bbfd9c043cbbb492542d68a3a81: Generated new TSK 0
I20260811 00:25:13.584329 23878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:13.598984 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23738
I20260811 00:25:13.600773 23836 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:13.694095 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23738
2026-08-11T00:25:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:13 dist-test-slave-f7th krb5kdc[23899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:13 dist-test-slave-f7th krb5kdc[23899](info): set up 2 sockets
Aug 11 00:25:13 dist-test-slave-f7th krb5kdc[23899](info): commencing operation
krb5kdc: starting...
W20260811 00:25:15.048488 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:15 dist-test-slave-f7th krb5kdc[23899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:15.422324 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40611
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:40611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:15.804849 23919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:15.805187 23919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:15.805307 23919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:15.813061 23919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:15.813202 23919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:15.813256 23919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:15.813326 23919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:15.813372 23919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:15.824012 23919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:40611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:40611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:15.825704 23919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:15.827207 23919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:15.843667 23924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:15.843554 23926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:15.843607 23928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:15.846196 23919 server_base.cc:1034] running on GCE node
I20260811 00:25:15.847332 23919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:15.849295 23919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:15.850634 23919 hybrid_clock.cc:648] HybridClock initialized: now 1786407915850580 us; error 49 us; skew 500 ppm
I20260811 00:25:15.851141 23919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:15 dist-test-slave-f7th krb5kdc[23899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407915, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:15.855366 23919 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:15.857383 23919 webserver.cc:460] Webserver started at http://127.16.71.190:37387/ using document root <none> and password file <none>
I20260811 00:25:15.858388 23919 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:15.858561 23919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:15.858940 23919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:15.862742 23919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "89679b587b954a0ea1a83ea044cc88d5"
format_stamp: "Formatted at 2026-08-11 00:25:15 on dist-test-slave-f7th"
server_key: "37a5d14861766f383c9646d26640fdc8"
server_key_iv: "320f8c2f3f4254406bcf8a0af816c997"
server_key_version: "encryptionkey@0"
I20260811 00:25:15.863684 23919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "89679b587b954a0ea1a83ea044cc88d5"
format_stamp: "Formatted at 2026-08-11 00:25:15 on dist-test-slave-f7th"
server_key: "37a5d14861766f383c9646d26640fdc8"
server_key_iv: "320f8c2f3f4254406bcf8a0af816c997"
server_key_version: "encryptionkey@0"
I20260811 00:25:15.870863 23919 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 00:25:15.875955 23941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:15.877799 23919 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:15.878058 23919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "89679b587b954a0ea1a83ea044cc88d5"
format_stamp: "Formatted at 2026-08-11 00:25:15 on dist-test-slave-f7th"
server_key: "37a5d14861766f383c9646d26640fdc8"
server_key_iv: "320f8c2f3f4254406bcf8a0af816c997"
server_key_version: "encryptionkey@0"
I20260811 00:25:15.878340 23919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:15.897675 23919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:15.903105 23919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:15.903558 23919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:15.924412 23919 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:40611
I20260811 00:25:15.924433 24022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:40611 every 8 connection(s)
I20260811 00:25:15.926000 23919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:15.930495 24023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:15.931176 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 23919
I20260811 00:25:15.931332 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:15.931809 16670 external_mini_cluster.cc:1428] Setting key 1d8ffb624b5c451216bc6cf84c6ad7e2
I20260811 00:25:15.946184 24023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5: Bootstrap starting.
I20260811 00:25:15.948151 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:15.934019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59867 (local address 127.16.71.190:40611)
0811 00:25:15.934915 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:15.934928 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:15.935001 (+ 73us) server_negotiation.cc:407] Connection header received
0811 00:25:15.937534 (+ 2533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:15.937574 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:15.938187 (+ 613us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:15.938631 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:15.939333 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:15.940768 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:15.942741 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:15.943649 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:15.943813 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:15.945393 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:15.945403 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:15.945421 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:15.946095 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:15.946195 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:15.946226 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:15.946487 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:15.946679 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 00:25:15.947026 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":409,"threads_started":1}
I20260811 00:25:15.950197 24023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:15.951537 24023 log.cc:824] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:15.952522 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:15.954738 24023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5: No bootstrap required, opened a new log
I20260811 00:25:15.963627 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:15.957045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43083 (local address 127.16.71.190:40611)
0811 00:25:15.957206 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:15.957212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:15.957807 (+ 595us) server_negotiation.cc:407] Connection header received
0811 00:25:15.958006 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:15.958014 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:15.958083 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:15.958181 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:15.958751 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:15.959524 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:15.960962 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:15.961465 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:15.961593 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:15.962854 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:15.962859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:15.962862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:15.962885 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:15.962943 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:15.962960 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:15.963080 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:15.963230 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 00:25:15.963328 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 00:25:15.963663 24023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89679b587b954a0ea1a83ea044cc88d5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40611 } }
I20260811 00:25:15.964146 24023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:15.964273 24023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89679b587b954a0ea1a83ea044cc88d5, State: Initialized, Role: FOLLOWER
I20260811 00:25:15.965059 24023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89679b587b954a0ea1a83ea044cc88d5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40611 } }
I20260811 00:25:15.965372 24023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:15.965525 24023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:15.965683 24023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:15.968407 24023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89679b587b954a0ea1a83ea044cc88d5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40611 } }
I20260811 00:25:15.969141 24023 leader_election.cc:302] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89679b587b954a0ea1a83ea044cc88d5; no voters:
I20260811 00:25:15.969743 24023 leader_election.cc:288] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:15.969885 24035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:15.970602 24035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [term 1 LEADER]: Becoming Leader. State: Replica: 89679b587b954a0ea1a83ea044cc88d5, State: Running, Role: LEADER
I20260811 00:25:15.971176 24035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89679b587b954a0ea1a83ea044cc88d5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40611 } }
I20260811 00:25:15.971889 24023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:15.974131 24037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89679b587b954a0ea1a83ea044cc88d5. Latest consensus state: current_term: 1 leader_uuid: "89679b587b954a0ea1a83ea044cc88d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89679b587b954a0ea1a83ea044cc88d5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40611 } } }
I20260811 00:25:15.974601 24037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:15.976804 24036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89679b587b954a0ea1a83ea044cc88d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89679b587b954a0ea1a83ea044cc88d5" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 40611 } } }
I20260811 00:25:15.977079 24036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:15.977614 24047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:15.982601 24047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:15.993499 24047 catalog_manager.cc:1347] Generated new cluster ID: 5078755be2d04a2997ab1299d56ea571
I20260811 00:25:15.993647 24047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:16.015805 24047 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:16.017616 24047 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:16.033334 24047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89679b587b954a0ea1a83ea044cc88d5: Generated new TSK 0
I20260811 00:25:16.034595 24047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:16.047159 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23919
W20260811 00:25:16.048542 24016 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59867
W20260811 00:25:16.049304 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 23919: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:16.049453 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 23919
2026-08-11T00:25:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:16 dist-test-slave-f7th krb5kdc[24065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:16 dist-test-slave-f7th krb5kdc[24065](info): set up 2 sockets
Aug 11 00:25:16 dist-test-slave-f7th krb5kdc[24065](info): commencing operation
krb5kdc: starting...
W20260811 00:25:17.370548 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:17 dist-test-slave-f7th krb5kdc[24065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:17.728441 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43341
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:44323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:43341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:18.110122 24091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:18.110462 24091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:18.110591 24091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:18.118445 24091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:18.118597 24091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:18.118654 24091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:18.118749 24091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:18.118808 24091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:18.129367 24091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:44323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:43341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:18.130800 24091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:18.132238 24091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:18.145278 24109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:18.145340 24104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:18.145331 24101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:18.146258 24091 server_base.cc:1034] running on GCE node
I20260811 00:25:18.147167 24091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:18.148767 24091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:18.150012 24091 hybrid_clock.cc:648] HybridClock initialized: now 1786407918149954 us; error 48 us; skew 500 ppm
I20260811 00:25:18.150463 24091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:18 dist-test-slave-f7th krb5kdc[24065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407918, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:18.154013 24091 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:18.155751 24091 webserver.cc:460] Webserver started at http://127.16.71.190:35745/ using document root <none> and password file <none>
I20260811 00:25:18.156530 24091 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:18.156706 24091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:18.157044 24091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:18.160403 24091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "94302b6a345f41428def8825b20e106f"
format_stamp: "Formatted at 2026-08-11 00:25:18 on dist-test-slave-f7th"
server_key: "48934738be2573fd07f1904142c0e5e3"
server_key_iv: "40f7f7c5e4cc258c125ae78336479bb6"
server_key_version: "encryptionkey@0"
I20260811 00:25:18.161329 24091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "94302b6a345f41428def8825b20e106f"
format_stamp: "Formatted at 2026-08-11 00:25:18 on dist-test-slave-f7th"
server_key: "48934738be2573fd07f1904142c0e5e3"
server_key_iv: "40f7f7c5e4cc258c125ae78336479bb6"
server_key_version: "encryptionkey@0"
I20260811 00:25:18.167335 24091 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:25:18.171705 24120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:18.173543 24091 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:18.173779 24091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "94302b6a345f41428def8825b20e106f"
format_stamp: "Formatted at 2026-08-11 00:25:18 on dist-test-slave-f7th"
server_key: "48934738be2573fd07f1904142c0e5e3"
server_key_iv: "40f7f7c5e4cc258c125ae78336479bb6"
server_key_version: "encryptionkey@0"
I20260811 00:25:18.174026 24091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:18.181082 24091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:18.186304 24091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:18.186659 24091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:18.206888 24091 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:43341
I20260811 00:25:18.206945 24201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:43341 every 8 connection(s)
I20260811 00:25:18.208453 24091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:18.212497 24202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:18.215230 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24091
I20260811 00:25:18.215416 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:18.215876 16670 external_mini_cluster.cc:1428] Setting key 62b96d12940f59d72ddbba6b68eacfc9
I20260811 00:25:18.225955 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:18.218132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53951 (local address 127.16.71.190:43341)
0811 00:25:18.219169 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:18.219187 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:18.219244 (+ 57us) server_negotiation.cc:407] Connection header received
0811 00:25:18.221640 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:18.221676 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:18.222164 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:18.222683 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:18.223782 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:18.223791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:18.223811 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:18.223829 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:18.223909 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:18.223938 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:18.224194 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:18.224345 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 00:25:18.224574 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":469,"threads_started":1}
I20260811 00:25:18.227177 24202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f: Bootstrap starting.
I20260811 00:25:18.230257 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:18.230855 24202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:18.231997 24202 log.cc:824] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:18.235935 24202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f: No bootstrap required, opened a new log
I20260811 00:25:18.237512 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:18.234947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52499 (local address 127.16.71.190:43341)
0811 00:25:18.235168 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:18.235174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:18.235508 (+ 334us) server_negotiation.cc:407] Connection header received
0811 00:25:18.235687 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:18.235691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:18.235742 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:18.235839 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:18.236801 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:18.236806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:18.236819 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:25:18.236837 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:18.236880 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:25:18.236896 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:18.237004 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:18.237132 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 00:25:18.237200 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:25:18.244797 24202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94302b6a345f41428def8825b20e106f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43341 } }
I20260811 00:25:18.245393 24202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:18.245512 24202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94302b6a345f41428def8825b20e106f, State: Initialized, Role: FOLLOWER
I20260811 00:25:18.246289 24202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94302b6a345f41428def8825b20e106f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43341 } }
I20260811 00:25:18.246554 24202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:18.246666 24202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:18.246800 24202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:18.249194 24202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94302b6a345f41428def8825b20e106f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43341 } }
I20260811 00:25:18.249737 24202 leader_election.cc:302] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94302b6a345f41428def8825b20e106f; no voters:
I20260811 00:25:18.250324 24202 leader_election.cc:288] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:18.250514 24216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:18.251204 24216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [term 1 LEADER]: Becoming Leader. State: Replica: 94302b6a345f41428def8825b20e106f, State: Running, Role: LEADER
I20260811 00:25:18.251835 24216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94302b6a345f41428def8825b20e106f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43341 } }
I20260811 00:25:18.252581 24202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:18.255144 24221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94302b6a345f41428def8825b20e106f. Latest consensus state: current_term: 1 leader_uuid: "94302b6a345f41428def8825b20e106f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94302b6a345f41428def8825b20e106f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43341 } } }
I20260811 00:25:18.255645 24221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:18.256124 24219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94302b6a345f41428def8825b20e106f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94302b6a345f41428def8825b20e106f" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43341 } } }
I20260811 00:25:18.256467 24219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:18.256651 24234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:18.267675 24234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:18.278909 24234 catalog_manager.cc:1347] Generated new cluster ID: d84a06d9b2f64049b259ecba3e6c554c
I20260811 00:25:18.279088 24234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:18.322773 24234 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:18.323282 24234 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:18.339964 24234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94302b6a345f41428def8825b20e106f: Generated new TSK 0
I20260811 00:25:18.341156 24234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:25:18.351714 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24091
W20260811 00:25:18.352991 24196 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53951
W20260811 00:25:18.353794 16670 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24091: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:18.353995 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24091
2026-08-11T00:25:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2313 ms)
[----------] 18 tests from AuthTokenIssuingTest (42384 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:18 dist-test-slave-f7th krb5kdc[24255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:18 dist-test-slave-f7th krb5kdc[24255](info): set up 2 sockets
Aug 11 00:25:18 dist-test-slave-f7th krb5kdc[24255](info): commencing operation
krb5kdc: starting...
W20260811 00:25:19.684801 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:19 dist-test-slave-f7th krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:20.049363 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45479
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:45479,127.16.71.189:34111,127.16.71.188:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:20.418727 24278 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:25:20.419123 24278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:20.419226 24278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:20.419353 24278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:20.427170 24278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:20.427289 24278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:20.427369 24278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:20.427435 24278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:20.427520 24278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:20.438014 24278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:45479,127.16.71.189:34111,127.16.71.188:36871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:20.439438 24278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:20.440968 24278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:20.453392 24286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:20.453513 24287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:20.453861 24291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:20.454869 24278 server_base.cc:1034] running on GCE node
I20260811 00:25:20.455865 24278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:20.457593 24278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:20.458876 24278 hybrid_clock.cc:648] HybridClock initialized: now 1786407920458804 us; error 34 us; skew 500 ppm
I20260811 00:25:20.459264 24278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:20 dist-test-slave-f7th krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407920, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:20.462812 24278 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:20.464556 24278 webserver.cc:460] Webserver started at http://127.16.71.190:36273/ using document root <none> and password file <none>
I20260811 00:25:20.465349 24278 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:20.465480 24278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:20.465816 24278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:20.469229 24278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "b3d86daae17641dfac0a83aa00c7fd42"
format_stamp: "Formatted at 2026-08-11 00:25:20 on dist-test-slave-f7th"
server_key: "cbd13288970809349d299744c99c5fb0"
server_key_iv: "8a3b9559aede6ca471fc01539c75dc01"
server_key_version: "encryptionkey@0"
I20260811 00:25:20.470180 24278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "b3d86daae17641dfac0a83aa00c7fd42"
format_stamp: "Formatted at 2026-08-11 00:25:20 on dist-test-slave-f7th"
server_key: "cbd13288970809349d299744c99c5fb0"
server_key_iv: "8a3b9559aede6ca471fc01539c75dc01"
server_key_version: "encryptionkey@0"
I20260811 00:25:20.476081 24278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:25:20.480258 24302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:20.482077 24278 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:20.482285 24278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "b3d86daae17641dfac0a83aa00c7fd42"
format_stamp: "Formatted at 2026-08-11 00:25:20 on dist-test-slave-f7th"
server_key: "cbd13288970809349d299744c99c5fb0"
server_key_iv: "8a3b9559aede6ca471fc01539c75dc01"
server_key_version: "encryptionkey@0"
I20260811 00:25:20.482546 24278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:20.497789 24278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:20.502303 24278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:20.502619 24278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:20.522158 24278 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:45479
I20260811 00:25:20.522178 24386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:45479 every 8 connection(s)
I20260811 00:25:20.523661 24278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:20.525286 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24278
I20260811 00:25:20.525586 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:20.526291 16670 external_mini_cluster.cc:1428] Setting key e1fb18a2bd22231eb703bd6ee3b6759a
I20260811 00:25:20.528133 24387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:20.535753 24387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:20 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407920, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.189@KRBTEST.COM; defaulting to no policy
I20260811 00:25:20.561156 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.539361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.16.71.190:45479)
0811 00:25:20.540332 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:20.540348 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:20.540940 (+ 592us) server_negotiation.cc:407] Connection header received
0811 00:25:20.543075 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:20.543082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:20.543405 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:20.543623 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:20.544932 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:20.546704 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:20.548133 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:20.549096 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:20.549284 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:20.553150 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:20.553159 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:20.553175 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:20.553210 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:20.556095 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:20.557191 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:20.557199 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:20.557201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:20.557374 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:20.557876 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:20.557880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:20.557882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:20.558455 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:20.558664 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:20.559549 (+ 885us) server_negotiation.cc:299] Negotiation successful
0811 00:25:20.560107 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":236,"server-negotiator.queue_time_us":704,"spinlock_wait_cycles":17152,"thread_start_us":409,"threads_started":1}
I20260811 00:25:20.561273 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.539361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45479 (local address 127.0.0.1:45082)
0811 00:25:20.540736 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.540793 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:20.542645 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:20.543687 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:20.543693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:20.544307 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:20.544709 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:20.544730 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:20.546880 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:20.546884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:20.547973 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:20.547986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:20.549348 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:20.549354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:20.549495 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:20.550151 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:20.550175 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:20.552861 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:20.556302 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:20.556319 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:20.556335 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:20.557028 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:20.557508 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:20.557514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:20.557517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:20.557755 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:20.558628 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:20.558637 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:20.558980 (+ 343us) client_negotiation.cc:769] Sending connection context
0811 00:25:20.559309 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 00:25:20.559921 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"mutex_wait_us":1,"thread_start_us":817,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.189@KRBTEST.COM" created.
I20260811 00:25:20.565034 24387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:25:20.566422 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.565939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:35096)
0811 00:25:20.566151 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.566281 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 00:25:20.566843 24306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
I20260811 00:25:20.568580 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.568159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:35108)
0811 00:25:20.568335 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.568416 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 00:25:20.568814 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:34111: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab.
Entry for principal kudu/127.16.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab.
I20260811 00:25:20.604602 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:34111
--webserver_interface=127.16.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab
--principal=kudu/127.16.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:45479,127.16.71.189:34111,127.16.71.188:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:25:20.618222 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } attempt: 1
I20260811 00:25:20.619720 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.619175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:35116)
0811 00:25:20.619437 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.619575 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:25:20.621026 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.620601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:35126)
0811 00:25:20.620774 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.620857 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 00:25:20.621237 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:34111: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
I20260811 00:25:20.689495 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } attempt: 2
I20260811 00:25:20.690694 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.690213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49714)
0811 00:25:20.690455 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.690552 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 00:25:20.691771 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.691360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49720)
0811 00:25:20.691538 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.691633 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 00:25:20.692000 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:34111: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
I20260811 00:25:20.783279 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } attempt: 3
I20260811 00:25:20.784556 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.784101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49730)
0811 00:25:20.784302 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.784398 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 00:25:20.785784 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.785332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49742)
0811 00:25:20.785541 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.785621 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260811 00:25:20.786013 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:34111: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
I20260811 00:25:20.929297 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } attempt: 4
I20260811 00:25:20.930661 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.930050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49746)
0811 00:25:20.930397 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.930495 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260811 00:25:20.931928 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:20.931459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49756)
0811 00:25:20.931621 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:20.931717 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 00:25:20.932169 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:34111: Network error: Client connection negotiation failed: client connection to 127.16.71.189:34111: connect: Connection refused (error 111)
W20260811 00:25:20.982445 24399 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:25:20.982820 24399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:20.982923 24399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:20.983048 24399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:20.991535 24399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:20.991669 24399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:20.991741 24399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:20.991818 24399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:20.991906 24399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:21.002965 24399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:45479,127.16.71.189:34111,127.16.71.188:36871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab
--principal=kudu/127.16.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:21.004531 24399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:21.006049 24399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:21.019187 24409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:21.019160 24413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:21.019122 24411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:21.019533 24399 server_base.cc:1034] running on GCE node
I20260811 00:25:21.020778 24399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:21.022480 24399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:21.023788 24399 hybrid_clock.cc:648] HybridClock initialized: now 1786407921023713 us; error 50 us; skew 500 ppm
I20260811 00:25:21.024187 24399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:21.027842 24399 init.cc:377] Logged in from keytab as kudu/127.16.71.189@KRBTEST.COM (short username kudu)
I20260811 00:25:21.029673 24399 webserver.cc:460] Webserver started at http://127.16.71.189:42097/ using document root <none> and password file <none>
I20260811 00:25:21.030455 24399 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:21.030573 24399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:21.030894 24399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:21.034540 24399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data/instance:
uuid: "fb40aaee49e4412f9654213f0bcfb33f"
format_stamp: "Formatted at 2026-08-11 00:25:21 on dist-test-slave-f7th"
server_key: "4ef9ea5cd839c83333402def9f6e7ebc"
server_key_iv: "70669d0d7e978d59afa5a69b6eec136b"
server_key_version: "encryptionkey@0"
I20260811 00:25:21.035471 24399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/wal/instance:
uuid: "fb40aaee49e4412f9654213f0bcfb33f"
format_stamp: "Formatted at 2026-08-11 00:25:21 on dist-test-slave-f7th"
server_key: "4ef9ea5cd839c83333402def9f6e7ebc"
server_key_iv: "70669d0d7e978d59afa5a69b6eec136b"
server_key_version: "encryptionkey@0"
I20260811 00:25:21.041989 24399 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:25:21.046470 24427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:21.048270 24399 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:25:21.048506 24399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/wal
uuid: "fb40aaee49e4412f9654213f0bcfb33f"
format_stamp: "Formatted at 2026-08-11 00:25:21 on dist-test-slave-f7th"
server_key: "4ef9ea5cd839c83333402def9f6e7ebc"
server_key_iv: "70669d0d7e978d59afa5a69b6eec136b"
server_key_version: "encryptionkey@0"
I20260811 00:25:21.048831 24399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:21.076095 24399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:21.081087 24399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:21.081377 24399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:21.101348 24399 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.189:34111
I20260811 00:25:21.101356 24501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.189:34111 every 8 connection(s)
I20260811 00:25:21.102846 24399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
I20260811 00:25:21.103242 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24399
I20260811 00:25:21.103443 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-1/wal/instance
I20260811 00:25:21.104051 16670 external_mini_cluster.cc:1428] Setting key 64d3c076f213e219196a07c5b5445496
I20260811 00:25:21.107477 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:21.115171 24502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.188@KRBTEST.COM; defaulting to no policy
I20260811 00:25:21.140220 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.118499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47626 (local address 127.16.71.190:45479)
0811 00:25:21.119237 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.119244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.120543 (+ 1299us) server_negotiation.cc:407] Connection header received
0811 00:25:21.122894 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.122901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.123000 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.123105 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.125045 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.125866 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.127900 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.128368 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.128506 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.132873 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.132880 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.132883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.132962 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.136183 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.137239 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.137245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.137247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.137411 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.137908 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.137912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.137914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.138298 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.138465 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.139396 (+ 931us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.139741 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":368,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.188@KRBTEST.COM" created.
I20260811 00:25:21.141290 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.119303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45479 (local address 127.0.0.1:47626)
0811 00:25:21.120310 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.120376 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.122519 (+ 2143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.123429 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.123439 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.124057 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.124768 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.124790 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.126067 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.126072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.127672 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.127691 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.128700 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.128711 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.128971 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.129764 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.129790 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.132518 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.136428 (+ 3910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.136438 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.136457 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.137063 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.137543 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.137546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.137549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.137766 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.138509 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.138517 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.138963 (+ 446us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.139246 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.139880 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":390,"threads_started":1}
I20260811 00:25:21.143849 24502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:25:21.159592 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.144867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49772)
0811 00:25:21.145225 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.145243 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.145355 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.146526 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.146531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.147026 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.147180 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.147189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.148581 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.148585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.149444 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.149457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.150363 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.150369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.150581 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.151222 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.151243 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.153774 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.157062 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.157067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.157072 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.157458 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.157841 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.157846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.157848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.157966 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.158607 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.158612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.158748 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.158868 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.159157 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260811 00:25:21.159636 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.145334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49772 (local address 127.16.71.189:34111)
0811 00:25:21.146058 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.146065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.146092 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:25:21.146145 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.146151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.146355 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.146511 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.147354 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.148438 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.149580 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.150162 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.150314 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.153995 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.154009 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.154025 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.154056 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.156924 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.157597 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.157605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.157607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.157723 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.158073 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.158077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.158079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.158466 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.158676 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.159039 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.159260 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":408,"threads_started":1}
I20260811 00:25:21.162375 24502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:25:21.163786 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.163230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51710)
0811 00:25:21.163454 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.163610 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260811 00:25:21.164106 24430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.165738 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.165376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51726)
0811 00:25:21.165524 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.165616 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 00:25:21.165952 24502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab.
Entry for principal kudu/127.16.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab.
I20260811 00:25:21.180888 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:36871
--webserver_interface=127.16.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab
--principal=kudu/127.16.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:45479,127.16.71.189:34111,127.16.71.188:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:25:21.215306 24502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 1
I20260811 00:25:21.216600 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.216106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51740)
0811 00:25:21.216317 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.216436 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 00:25:21.217698 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.217318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51748)
0811 00:25:21.217475 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.217574 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:25:21.217954 24502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.231442 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } attempt: 5
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407920, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:25:21.248265 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.232259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49780)
0811 00:25:21.232493 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.232526 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.232700 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.233152 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.233155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.233592 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.233788 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.233798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.235245 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.235249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.236156 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.236171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.237244 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.237248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.237412 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.238028 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.238047 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.240443 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.244676 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.244683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.244687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.245209 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.245754 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.245764 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.245768 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.246046 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.246907 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.246914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.247185 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.247363 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.247874 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 00:25:21.248865 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.232474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49780 (local address 127.16.71.189:34111)
0811 00:25:21.232717 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.232723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.232777 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:25:21.232856 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.232861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.232972 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.233095 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.233962 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.235066 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.236323 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.237073 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.237216 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.240729 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.240736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.240740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.240788 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.244408 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.245373 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.245380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.245384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.245572 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.246210 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.246250 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.246254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.246702 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.246881 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.247554 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.247947 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:25:21.250214 24387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:25:21.251899 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.251400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51762)
0811 00:25:21.251601 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.251706 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 00:25:21.253226 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.252699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51774)
0811 00:25:21.252879 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.253088 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 00:25:21.253588 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.286221 24502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 2
I20260811 00:25:21.287639 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.287074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51786)
0811 00:25:21.287305 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.287460 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 00:25:21.288897 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.288472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51800)
0811 00:25:21.288663 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.288749 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 00:25:21.289119 24502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.304823 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 1
I20260811 00:25:21.306164 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.305597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51806)
0811 00:25:21.305820 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.305938 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 00:25:21.307389 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.307001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51810)
0811 00:25:21.307160 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.307250 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:25:21.307633 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.375907 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 2
I20260811 00:25:21.377359 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.376745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51822)
0811 00:25:21.376991 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.377146 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:25:21.378595 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.378172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51836)
0811 00:25:21.378355 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.378447 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:25:21.378886 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.380345 24502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 3
I20260811 00:25:21.381518 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.381045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51850)
0811 00:25:21.381260 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.381347 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:25:21.382602 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.382136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51858)
0811 00:25:21.382398 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.382484 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 00:25:21.382833 24502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.485160 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 3
I20260811 00:25:21.486480 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.485877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51872)
0811 00:25:21.486123 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.486240 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 00:25:21.487833 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.487323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51874)
0811 00:25:21.487576 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.487679 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 00:25:21.488148 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.526082 24502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 4
I20260811 00:25:21.527314 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.526762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51886)
0811 00:25:21.527058 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.527151 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260811 00:25:21.528558 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.528173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51894)
0811 00:25:21.528338 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.528419 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 00:25:21.529053 24502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
W20260811 00:25:21.554437 24517 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:25:21.554800 24517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:21.554879 24517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:21.554962 24517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:21.562600 24517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:21.562717 24517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:21.562776 24517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:21.562827 24517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:21.562891 24517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:21.573225 24517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:45479,127.16.71.189:34111,127.16.71.188:36871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab
--principal=kudu/127.16.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:36871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:21.574584 24517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:21.575932 24517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:21.588317 24533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:21.588349 24536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:21.588408 24538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:21.589206 24517 server_base.cc:1034] running on GCE node
I20260811 00:25:21.590092 24517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:21.591701 24517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:21.592942 24517 hybrid_clock.cc:648] HybridClock initialized: now 1786407921592857 us; error 56 us; skew 500 ppm
I20260811 00:25:21.593339 24517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:21.596782 24517 init.cc:377] Logged in from keytab as kudu/127.16.71.188@KRBTEST.COM (short username kudu)
I20260811 00:25:21.598441 24517 webserver.cc:460] Webserver started at http://127.16.71.188:33857/ using document root <none> and password file <none>
I20260811 00:25:21.599160 24517 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:21.599253 24517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:21.599541 24517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:21.602671 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data/instance:
uuid: "9ddefe9030a148e59621584474d2aa0f"
format_stamp: "Formatted at 2026-08-11 00:25:21 on dist-test-slave-f7th"
server_key: "26b17d32218d5c679824f9061dc3ca3c"
server_key_iv: "0df47b6180dac887279def6ef111e912"
server_key_version: "encryptionkey@0"
I20260811 00:25:21.603530 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/wal/instance:
uuid: "9ddefe9030a148e59621584474d2aa0f"
format_stamp: "Formatted at 2026-08-11 00:25:21 on dist-test-slave-f7th"
server_key: "26b17d32218d5c679824f9061dc3ca3c"
server_key_iv: "0df47b6180dac887279def6ef111e912"
server_key_version: "encryptionkey@0"
I20260811 00:25:21.609277 24517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 00:25:21.613274 24547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:21.614966 24517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:25:21.615165 24517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/wal
uuid: "9ddefe9030a148e59621584474d2aa0f"
format_stamp: "Formatted at 2026-08-11 00:25:21 on dist-test-slave-f7th"
server_key: "26b17d32218d5c679824f9061dc3ca3c"
server_key_iv: "0df47b6180dac887279def6ef111e912"
server_key_version: "encryptionkey@0"
I20260811 00:25:21.615401 24517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:21.644958 24517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:21.650065 24517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:21.650357 24517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:21.665406 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 4
I20260811 00:25:21.666687 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.666115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51906)
0811 00:25:21.666380 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.666503 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 00:25:21.668032 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.667559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51916)
0811 00:25:21.667745 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.667849 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 00:25:21.668351 24387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:36871: Network error: Client connection negotiation failed: client connection to 127.16.71.188:36871: connect: Connection refused (error 111)
I20260811 00:25:21.670478 24517 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.188:36871
I20260811 00:25:21.670459 24634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.188:36871 every 8 connection(s)
I20260811 00:25:21.671962 24517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
I20260811 00:25:21.676170 24635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:21.676199 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24517
I20260811 00:25:21.676400 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407527335556-16670-0/minicluster-data/master-2/wal/instance
I20260811 00:25:21.677026 16670 external_mini_cluster.cc:1428] Setting key 0c9b57180ba7764db20ed32c37e9e016
I20260811 00:25:21.686332 24635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:25:21.704267 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.679103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.16.71.188:36871)
0811 00:25:21.680088 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.680102 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.680150 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:25:21.682807 (+ 2657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.682839 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.683368 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.683780 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.685857 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.687414 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.689268 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.690444 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.690698 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.695878 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.695887 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.695900 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.695990 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.699474 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.700328 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.700335 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.700343 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.700508 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.700959 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.700963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.700965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.701542 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.701851 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.702356 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.702844 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":438,"threads_started":1}
I20260811 00:25:21.705638 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.689487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.16.71.190:45479)
0811 00:25:21.690031 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.690038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.690503 (+ 465us) server_negotiation.cc:407] Connection header received
0811 00:25:21.690747 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.690753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.690852 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.690967 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.692187 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.693346 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.695156 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.695789 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.695915 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.699844 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.699849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.699853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.699898 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.702539 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.703413 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.703417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.703420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.703559 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.704010 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.704014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.704017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.704324 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.704514 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.705142 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.705334 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":234,"threads_started":1}
I20260811 00:25:21.705708 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.689514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:45479 (local address 127.0.0.1:47638)
0811 00:25:21.690333 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.690380 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.690576 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.691073 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.691079 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.691698 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.692005 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.692016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.693554 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.693559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.694939 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.694992 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.696011 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.696018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.696221 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.697037 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.697055 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.699615 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.702714 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.702725 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.702731 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.703258 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.703678 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.703683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.703686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.703844 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.704494 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.704502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.704769 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.704980 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.705301 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":453,"threads_started":1}
I20260811 00:25:21.708338 24635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:25:21.723208 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.709270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:34111 (local address 127.0.0.1:49782)
0811 00:25:21.709456 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.709475 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.709570 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.710817 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.710821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.711182 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.711337 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.711346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.712549 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.712554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.713550 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.713565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.714515 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.714519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.714652 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.715246 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.715266 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.717573 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.720681 (+ 3108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.720687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.720692 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.721110 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.721530 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.721535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.721539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.721694 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.722223 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.722228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.722418 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.722552 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.722861 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 00:25:21.723209 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.707705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.16.71.190:45479)
0811 00:25:21.707888 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.707894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.707978 (+ 84us) server_negotiation.cc:407] Connection header received
0811 00:25:21.708204 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.708208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.708281 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.708369 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.709935 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.710646 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.712084 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.712520 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.712700 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.717574 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.717581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.717585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.717632 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.720515 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.721294 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.721298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.721301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.721439 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.721820 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.721824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.721826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.722109 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.722387 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.722653 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.722887 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:25:21.723271 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.709291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.16.71.189:34111)
0811 00:25:21.709496 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.709501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.709531 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:25:21.709677 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.709682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.709759 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.709866 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.711500 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.712380 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.713721 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.714360 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.714501 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.717780 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.717785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.717788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.717824 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.720488 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.721245 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.721251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.721254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.721414 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.721809 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.721812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.721815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.722094 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.722239 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.722681 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.722909 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:25:21.724819 24635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:25:21.738617 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.725671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51934)
0811 00:25:21.725830 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.725844 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.725921 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.726275 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.726279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.726681 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.726834 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.726843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.727822 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.727827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.728737 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.728750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.729507 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.729514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.729739 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.730351 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.730377 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.732773 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.736128 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.736134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.736139 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.736517 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.736887 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.736900 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.736904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.737030 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.737621 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.737624 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.737762 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.737936 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.738268 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 00:25:21.738873 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.725899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.16.71.188:36871)
0811 00:25:21.726065 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.726069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.726081 (+ 12us) server_negotiation.cc:407] Connection header received
0811 00:25:21.726114 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.726117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.726165 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.726243 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.726981 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.727691 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.728872 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.729313 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.729421 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.733004 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.733009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.733012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.733044 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.735969 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.736655 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.736659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.736661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.736785 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.737129 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.737135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.737138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.737485 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.737639 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.738056 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.738469 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":54}
I20260811 00:25:21.740777 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.724962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49784 (local address 127.16.71.189:34111)
0811 00:25:21.725338 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.725342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.725357 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:25:21.725404 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.725408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.725481 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.725580 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.726957 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.727682 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.728986 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.729485 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.729617 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.734487 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.734493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.734496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.734533 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.737251 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.738083 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.738088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.738091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.738268 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.738705 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.738710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.738713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.739548 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.739838 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.740157 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.740408 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260811 00:25:21.744441 24635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: Bootstrap starting.
I20260811 00:25:21.745779 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:21.748186 24635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:21.749425 24635 log.cc:824] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:21.748829 24349 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b3d86daae17641dfac0a83aa00c7fd42"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47642
I20260811 00:25:21.752794 24635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: No bootstrap required, opened a new log
I20260811 00:25:21.760345 24635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:21.760998 24635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ddefe9030a148e59621584474d2aa0f, State: Initialized, Role: FOLLOWER
I20260811 00:25:21.761546 24635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:21.762812 24646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:21.763329 24646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:21.763904 24635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:25:21.774421 24664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:25:21.774600 24664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:25:21.772240 24517 master_runner.cc:186] Error getting master registration for 127.16.71.190:45479: OK, instance_id { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" instance_seqno: 1786407920516816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:25:21.783977 24517 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:25:21.828284 24502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 5
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407921, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
I20260811 00:25:21.845013 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.829094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51942)
0811 00:25:21.829304 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.829336 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.829560 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.830021 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.830026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.830501 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.830695 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.830710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.831882 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.831888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.832858 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.832874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.833677 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.833683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.833918 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.834670 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.834696 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.837572 (+ 2876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.842009 (+ 4437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.842014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.842017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.842496 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.842949 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.842954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.842958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.843143 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.843885 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.843889 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.844058 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.844219 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.844547 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 00:25:21.845014 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.829227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.16.71.188:36871)
0811 00:25:21.829546 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.829554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.829583 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:25:21.829721 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.829726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.829816 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.829975 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.830891 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.831710 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.833017 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.833512 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.833634 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.837859 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.837871 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.837877 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.837937 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.841808 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.842636 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.842642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.842645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.842821 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.843262 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.843267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.843270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.843754 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.843963 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.844344 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.844600 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:25:21.851318 24502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: Bootstrap starting.
I20260811 00:25:21.855180 24502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:21.856355 24502 log.cc:824] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:21.859432 24502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: No bootstrap required, opened a new log
I20260811 00:25:21.867589 24502 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:21.867964 24502 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb40aaee49e4412f9654213f0bcfb33f, State: Initialized, Role: FOLLOWER
I20260811 00:25:21.868475 24502 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:21.869498 24669 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:21.869913 24669 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:21.870786 24502 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:25:21.881676 24695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:25:21.881958 24695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:25:21.878582 24399 master_runner.cc:186] Error getting master registration for 127.16.71.190:45479: OK, instance_id { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" instance_seqno: 1786407920516816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:25:21.890924 24399 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 00:25:21.902763 24553 tablet.cc:2369] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:21.945632 24387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } attempt: 5
Aug 11 00:25:21 dist-test-slave-f7th krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407920, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
I20260811 00:25:21.962944 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.946471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.16.71.188:36871)
0811 00:25:21.946662 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:21.946668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:21.946799 (+ 131us) server_negotiation.cc:407] Connection header received
0811 00:25:21.947004 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:21.947009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:21.947093 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:21.947190 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:21.948396 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.949699 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.951096 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.951928 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.952067 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.956411 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:21.956416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:21.956420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:21.956460 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:21.959647 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.960594 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.960600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.960602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.960805 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:21.961311 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:21.961316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:21.961319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:21.961647 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:21.961794 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:21.962310 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 00:25:21.962559 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:25:21.962961 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:21.946413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:36871 (local address 127.0.0.1:51952)
0811 00:25:21.946703 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:21.946722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:21.946874 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:21.947236 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:21.947240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:21.947971 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:21.948165 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.948177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.949886 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.949893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.950903 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:21.950921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:21.952100 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:21.952106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:21.952342 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:21.953113 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:21.953141 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:21.956147 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:21.959845 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.959852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.959857 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.960455 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.960953 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:21.960958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:21.960963 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:21.961178 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:21.961798 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:21.961804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:21.962016 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 00:25:21.962173 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 00:25:21.962528 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260811 00:25:21.970546 24387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42: Bootstrap starting.
I20260811 00:25:21.975492 24387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:21.977012 24387 log.cc:824] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:21.980906 24387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42: No bootstrap required, opened a new log
I20260811 00:25:21.989184 24387 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:21.989754 24387 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d86daae17641dfac0a83aa00c7fd42, State: Initialized, Role: FOLLOWER
I20260811 00:25:21.990336 24387 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:21.991763 24700 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:21.992563 24700 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:21.993192 24387 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:21.993297 24349 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 00:25:21.993507 24349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:21.998446 24349 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:22.001965 24464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3d86daae17641dfac0a83aa00c7fd42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb40aaee49e4412f9654213f0bcfb33f"
I20260811 00:25:22.002673 24464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [term 0 FOLLOWER]: Advancing to term 1
W20260811 00:25:22.006163 24309 tablet.cc:2369] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:22.007882 24464 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3d86daae17641dfac0a83aa00c7fd42 in term 1.
I20260811 00:25:22.009032 24306 leader_election.cc:302] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3d86daae17641dfac0a83aa00c7fd42, fb40aaee49e4412f9654213f0bcfb33f; no voters:
I20260811 00:25:22.011633 24597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3d86daae17641dfac0a83aa00c7fd42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9ddefe9030a148e59621584474d2aa0f"
I20260811 00:25:22.012275 24597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:22.017385 24597 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3d86daae17641dfac0a83aa00c7fd42 in term 1.
I20260811 00:25:22.018610 24349 leader_election.cc:288] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [CANDIDATE]: Term 1 election: Requested vote from peers fb40aaee49e4412f9654213f0bcfb33f (127.16.71.189:34111), 9ddefe9030a148e59621584474d2aa0f (127.16.71.188:36871)
I20260811 00:25:22.026759 24700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:22.038411 24700 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [term 1 LEADER]: Becoming Leader. State: Replica: b3d86daae17641dfac0a83aa00c7fd42, State: Running, Role: LEADER
I20260811 00:25:22.039258 24700 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } }
I20260811 00:25:22.059031 24716 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3d86daae17641dfac0a83aa00c7fd42. Latest consensus state: current_term: 1 leader_uuid: "b3d86daae17641dfac0a83aa00c7fd42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:22.059672 24716 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:22.069591 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:22.029744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.16.71.189:34111)
0811 00:25:22.029959 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.029966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.039301 (+ 9335us) server_negotiation.cc:407] Connection header received
0811 00:25:22.039852 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.039870 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.039970 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.040046 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.044424 (+ 4378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.045622 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.049447 (+ 3825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.050182 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.050291 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:22.059995 (+ 9704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:22.060005 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:22.060011 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:22.060068 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:22.064456 (+ 4388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:22.065585 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:22.065592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:22.065595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:22.065797 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:22.066679 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:22.066685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:22.066689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:22.067277 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:22.067571 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:22.068843 (+ 1272us) server_negotiation.cc:299] Negotiation successful
0811 00:25:22.069196 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:25:22.078606 24731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:22.083730 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:22.026914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.16.71.190:45479)
0811 00:25:22.027149 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.027157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.029062 (+ 1905us) server_negotiation.cc:407] Connection header received
0811 00:25:22.030747 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.030755 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.030860 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.030946 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.043031 (+ 12085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.044320 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.050070 (+ 5750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.050727 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.050832 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:22.063971 (+ 13139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:22.063982 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:22.063988 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:22.064047 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:22.068369 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:22.076714 (+ 8345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:22.076723 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:22.076728 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:22.076985 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:22.079055 (+ 2070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:22.079063 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:22.079067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:22.079578 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:22.079932 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:22.082897 (+ 2965us) server_negotiation.cc:299] Negotiation successful
0811 00:25:22.083268 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 00:25:22.084468 24434 tablet.cc:2369] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:22.087502 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:22.027934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.16.71.188:36871)
0811 00:25:22.028208 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.028215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.030451 (+ 2236us) server_negotiation.cc:407] Connection header received
0811 00:25:22.034058 (+ 3607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.034066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.034201 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.034280 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.046495 (+ 12215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.047583 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.051695 (+ 4112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.052797 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.052918 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:22.070557 (+ 17639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:22.070569 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:22.070574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:22.070623 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:22.075238 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:22.080597 (+ 5359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:22.080605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:22.080609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:22.080844 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:22.081393 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:22.081399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:22.081403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:22.081859 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:22.082191 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:22.086602 (+ 4411us) server_negotiation.cc:299] Negotiation successful
0811 00:25:22.086989 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 00:25:22.108450 24731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:22.154825 24464 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [term 1 FOLLOWER]: Refusing update from remote peer b3d86daae17641dfac0a83aa00c7fd42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:22.157392 24597 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [term 1 FOLLOWER]: Refusing update from remote peer b3d86daae17641dfac0a83aa00c7fd42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:22.159106 24700 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:22.160581 24716 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:22.166565 24743 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:25:22.188230 24669 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3d86daae17641dfac0a83aa00c7fd42. Latest consensus state: current_term: 1 leader_uuid: "b3d86daae17641dfac0a83aa00c7fd42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:22.188575 24669 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:22.196358 24739 mvcc.cc:205] Tried to move back new op lower bound from 7317126849124470784 to 7317126848682094592. Current Snapshot: MvccSnapshot[applied={T|T < 7317126849124470784}]
I20260811 00:25:22.198562 24744 mvcc.cc:205] Tried to move back new op lower bound from 7317126849124470784 to 7317126848682094592. Current Snapshot: MvccSnapshot[applied={T|T < 7317126849124470784}]
I20260811 00:25:22.201148 24716 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3d86daae17641dfac0a83aa00c7fd42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:22.201499 24716 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:22.203734 24731 catalog_manager.cc:1347] Generated new cluster ID: 409752dea0f94b3fac57b2fcd0a4253d
I20260811 00:25:22.203871 24731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:22.216090 24669 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3d86daae17641dfac0a83aa00c7fd42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:22.216432 24669 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:22.231236 24646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3d86daae17641dfac0a83aa00c7fd42. Latest consensus state: current_term: 1 leader_uuid: "b3d86daae17641dfac0a83aa00c7fd42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:22.231653 24646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:22.242776 24700 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3d86daae17641dfac0a83aa00c7fd42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:22.243144 24700 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:22.252774 24646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3d86daae17641dfac0a83aa00c7fd42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d86daae17641dfac0a83aa00c7fd42" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 45479 } } peers { permanent_uuid: "fb40aaee49e4412f9654213f0bcfb33f" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 34111 } } peers { permanent_uuid: "9ddefe9030a148e59621584474d2aa0f" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 36871 } } }
I20260811 00:25:22.253088 24646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:22.260587 24731 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:22.263597 24731 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:22.285727 24731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3d86daae17641dfac0a83aa00c7fd42: Generated new TSK 0
I20260811 00:25:22.287370 24731 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:25:22.404304 24639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.389609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47670 (local address 127.16.71.190:45479)
0811 00:25:22.389838 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.389845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.394927 (+ 5082us) server_negotiation.cc:407] Connection header received
0811 00:25:22.402488 (+ 7561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.402496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.402594 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.402678 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.404016 (+ 1338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47670: BlockingRecv error: recv got EOF from 127.0.0.1:47670 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 00:25:22.406165 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.389657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.16.71.189:34111)
0811 00:25:22.389850 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.389857 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.402339 (+ 12482us) server_negotiation.cc:407] Connection header received
0811 00:25:22.403578 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.403584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.403695 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.403776 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.405922 (+ 2146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49790: BlockingRecv error: recv got EOF from 127.0.0.1:49790 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 00:25:22.407055 24638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.395424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.16.71.188:36871)
0811 00:25:22.395633 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.395639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.405116 (+ 9477us) server_negotiation.cc:407] Connection header received
0811 00:25:22.405387 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.405393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.405479 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.405556 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.406776 (+ 1220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51974: BlockingRecv error: recv got EOF from 127.0.0.1:51974 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 00:25:22.415932 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.413376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.16.71.189:34111)
0811 00:25:22.413563 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.413569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.414106 (+ 537us) server_negotiation.cc:407] Connection header received
0811 00:25:22.414348 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.414355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.414509 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.414637 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.415569 (+ 932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49802: BlockingRecv error: recv got EOF from 127.0.0.1:49802 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 00:25:22.429793 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.427324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.16.71.189:34111)
0811 00:25:22.427514 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.427520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.428078 (+ 558us) server_negotiation.cc:407] Connection header received
0811 00:25:22.428295 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.428301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.428421 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.428541 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.429550 (+ 1009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49814: BlockingRecv error: recv got EOF from 127.0.0.1:49814 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 00:25:22.441592 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.438834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49818 (local address 127.16.71.189:34111)
0811 00:25:22.439040 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.439047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.439901 (+ 854us) server_negotiation.cc:407] Connection header received
0811 00:25:22.440132 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.440139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.440257 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.440368 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.441309 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49818: BlockingRecv error: recv got EOF from 127.0.0.1:49818 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 00:25:22.451711 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.449185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.16.71.189:34111)
0811 00:25:22.449494 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.449501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.450135 (+ 634us) server_negotiation.cc:407] Connection header received
0811 00:25:22.450354 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.450360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.450459 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.450566 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.451471 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49830: BlockingRecv error: recv got EOF from 127.0.0.1:49830 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
W20260811 00:25:22.465956 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.463689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.16.71.189:34111)
0811 00:25:22.463955 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.463962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.464500 (+ 538us) server_negotiation.cc:407] Connection header received
0811 00:25:22.464717 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.464721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.464800 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.464890 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.465720 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49836: BlockingRecv error: recv got EOF from 127.0.0.1:49836 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 00:25:22.488445 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.486076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.16.71.189:34111)
0811 00:25:22.486318 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.486325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.486784 (+ 459us) server_negotiation.cc:407] Connection header received
0811 00:25:22.487054 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.487058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.487136 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.487226 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.488211 (+ 985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49846: BlockingRecv error: recv got EOF from 127.0.0.1:49846 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 00:25:22.527604 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.524771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.16.71.189:34111)
0811 00:25:22.524994 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.525000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.525895 (+ 895us) server_negotiation.cc:407] Connection header received
0811 00:25:22.526126 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.526132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.526232 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.526350 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.527320 (+ 970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49850: BlockingRecv error: recv got EOF from 127.0.0.1:49850 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 00:25:22.597661 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.595635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49852 (local address 127.16.71.189:34111)
0811 00:25:22.595855 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.595862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.596221 (+ 359us) server_negotiation.cc:407] Connection header received
0811 00:25:22.596415 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.596421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.596519 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.596671 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.597435 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49852: BlockingRecv error: recv got EOF from 127.0.0.1:49852 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 00:25:22.731534 24511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:22.729626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.16.71.189:34111)
0811 00:25:22.729874 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.729881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.730101 (+ 220us) server_negotiation.cc:407] Connection header received
0811 00:25:22.730314 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.730319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.730394 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.730484 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:22.731325 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49868: BlockingRecv error: recv got EOF from 127.0.0.1:49868 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:25:22.776894 24664 catalog_manager.cc:1259] Loaded cluster ID: 409752dea0f94b3fac57b2fcd0a4253d
I20260811 00:25:22.777006 24664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: loading cluster ID for follower catalog manager: success
I20260811 00:25:22.779609 24664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: acquiring CA information for follower catalog manager: success
I20260811 00:25:22.781910 24664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9ddefe9030a148e59621584474d2aa0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:25:22.884254 24695 catalog_manager.cc:1259] Loaded cluster ID: 409752dea0f94b3fac57b2fcd0a4253d
I20260811 00:25:22.884383 24695 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: loading cluster ID for follower catalog manager: success
I20260811 00:25:22.887125 24695 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: acquiring CA information for follower catalog manager: success
I20260811 00:25:22.889072 24695 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fb40aaee49e4412f9654213f0bcfb33f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:25:22.998657 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:22.992226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49874 (local address 127.16.71.189:34111)
0811 00:25:22.992452 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:22.992458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:22.992660 (+ 202us) server_negotiation.cc:407] Connection header received
0811 00:25:22.992809 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:22.992813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:22.992989 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:22.993136 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:22.993671 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.994636 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.996125 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:22.996564 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:22.996745 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:22.997330 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:22.997963 (+ 633us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:22.998091 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:22.998208 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 00:25:22.998342 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:25:23.012189 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:23.005537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.16.71.188:36871)
0811 00:25:23.006555 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:23.006559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:23.006575 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:25:23.006713 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:23.006717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:23.006895 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:23.007060 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:23.007505 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.008289 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.009822 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.010269 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.010378 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:23.010889 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:23.011514 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:23.011626 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:23.011741 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 00:25:23.011827 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":802,"threads_started":1}
I20260811 00:25:23.040591 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:23.021218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.16.71.188:36871)
0811 00:25:23.025974 (+ 4756us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:23.025982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:23.034353 (+ 8371us) server_negotiation.cc:407] Connection header received
0811 00:25:23.034508 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:23.034514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:23.034613 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:23.034711 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:23.035496 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.036529 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.038555 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.039013 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.039133 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:23.039737 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:23.039887 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:23.039971 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:23.040176 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 00:25:23.040272 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4525}
I20260811 00:25:23.040923 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:23.020860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.16.71.189:34111)
0811 00:25:23.021075 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:23.021082 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:23.033306 (+ 12224us) server_negotiation.cc:407] Connection header received
0811 00:25:23.034265 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:23.034272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:23.034379 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:23.034485 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:23.035116 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.036253 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.038678 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.039356 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.039497 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:23.040096 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:23.040249 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:23.040334 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:23.040503 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 00:25:23.040603 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:25:23.042696 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:23.020792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.16.71.190:45479)
0811 00:25:23.035475 (+ 14683us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:23.035481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:23.035499 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:25:23.035599 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:23.035603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:23.035753 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:23.035941 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:23.036485 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.037750 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.040053 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:23.040896 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:23.041049 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:23.041642 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:23.042016 (+ 374us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:23.042154 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:23.042319 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 00:25:23.042421 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14548,"thread_start_us":14413,"threads_started":1}
I20260811 00:25:23.047899 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24278
I20260811 00:25:23.049196 24380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:23.163153 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24278
I20260811 00:25:23.190985 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24399
I20260811 00:25:23.192198 24495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:23.305161 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24399
I20260811 00:25:23.336268 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24517
I20260811 00:25:23.337603 24630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:23.446105 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24517
2026-08-11T00:25:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5087 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:23 dist-test-slave-f7th krb5kdc[24882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:23 dist-test-slave-f7th krb5kdc[24882](info): set up 2 sockets
Aug 11 00:25:23 dist-test-slave-f7th krb5kdc[24882](info): commencing operation
krb5kdc: starting...
W20260811 00:25:24.788578 16670 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/test-user.keytab.
Aug 11 00:25:25 dist-test-slave-f7th krb5kdc[24882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
Entry for principal kudu/127.16.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab.
I20260811 00:25:25.160804 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43973
--webserver_interface=127.16.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:43973,127.16.71.189:36207,127.16.71.188:45289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:25.544473 24904 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:25:25.544932 24904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:25.545046 24904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:25.545171 24904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:25.552946 24904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:25.553073 24904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:25.553157 24904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:25.553223 24904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:25.553308 24904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:25.564028 24904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:43973,127.16.71.189:36207,127.16.71.188:45289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.190.keytab
--principal=kudu/127.16.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.190:43973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:25.565635 24904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:25.567198 24904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:25.580494 24913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:25.580521 24915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:25.580636 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:25.581663 24904 server_base.cc:1034] running on GCE node
I20260811 00:25:25.582680 24904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:25.584359 24904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:25.585719 24904 hybrid_clock.cc:648] HybridClock initialized: now 1786407925585610 us; error 64 us; skew 500 ppm
I20260811 00:25:25.586136 24904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:25 dist-test-slave-f7th krb5kdc[24882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:25.589933 24904 init.cc:377] Logged in from keytab as kudu/127.16.71.190@KRBTEST.COM (short username kudu)
I20260811 00:25:25.591786 24904 webserver.cc:460] Webserver started at http://127.16.71.190:42605/ using document root <none> and password file <none>
I20260811 00:25:25.592607 24904 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:25.592795 24904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:25.593159 24904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:25.596712 24904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data/instance:
uuid: "6c70aed3cdb340ada4e6f86f4857e72a"
format_stamp: "Formatted at 2026-08-11 00:25:25 on dist-test-slave-f7th"
server_key: "76e13913f396daf10fbae1dc2910529c"
server_key_iv: "e9711b2e5f12aced6f92572e2780e73d"
server_key_version: "encryptionkey@0"
I20260811 00:25:25.597667 24904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/wal/instance:
uuid: "6c70aed3cdb340ada4e6f86f4857e72a"
format_stamp: "Formatted at 2026-08-11 00:25:25 on dist-test-slave-f7th"
server_key: "76e13913f396daf10fbae1dc2910529c"
server_key_iv: "e9711b2e5f12aced6f92572e2780e73d"
server_key_version: "encryptionkey@0"
I20260811 00:25:25.603695 24904 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:25:25.607632 24928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:25.609318 24904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:25:25.609514 24904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/wal
uuid: "6c70aed3cdb340ada4e6f86f4857e72a"
format_stamp: "Formatted at 2026-08-11 00:25:25 on dist-test-slave-f7th"
server_key: "76e13913f396daf10fbae1dc2910529c"
server_key_iv: "e9711b2e5f12aced6f92572e2780e73d"
server_key_version: "encryptionkey@0"
I20260811 00:25:25.609746 24904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:25.630661 24904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:25.635771 24904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:25.636111 24904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:25.655598 24904 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.190:43973
I20260811 00:25:25.655648 25027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.190:43973 every 8 connection(s)
I20260811 00:25:25.657258 24904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:25.658710 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 24904
I20260811 00:25:25.658943 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-0/wal/instance
I20260811 00:25:25.659528 16670 external_mini_cluster.cc:1428] Setting key 5ccb1339d9bcf0db2590cbf6033a78b6
I20260811 00:25:25.662758 25028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:25.672564 25028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:25 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.71.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.189@KRBTEST.COM" created.
I20260811 00:25:25.700925 25034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.677222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57078 (local address 127.16.71.190:43973)
0811 00:25:25.679190 (+ 1968us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:25.679199 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:25.679262 (+ 63us) server_negotiation.cc:407] Connection header received
0811 00:25:25.681105 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:25.681117 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:25.681534 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:25.681773 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:25.683145 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:25.685034 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:25.686665 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:25.687623 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:25.687840 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:25.692202 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:25.692212 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:25.692229 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:25.692269 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:25.695583 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:25.696782 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:25.696791 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:25.696793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:25.696988 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:25.697441 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:25.697444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:25.697446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:25.698163 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:25.698363 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:25.699043 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 00:25:25.699417 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1749,"spinlock_wait_cycles":61696,"thread_start_us":1494,"threads_started":1}
I20260811 00:25:25.700995 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.677223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:43973 (local address 127.0.0.1:57078)
0811 00:25:25.678376 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.678463 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:25.680563 (+ 2100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:25.681799 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:25.681805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:25.682482 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:25.682878 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:25.682900 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:25.685234 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:25.685239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:25.686474 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:25.686492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:25.687927 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:25.687935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:25.688084 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:25.688835 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:25.688862 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:25.691802 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:25.695820 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:25.695839 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:25.695866 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:25.696563 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:25.697132 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:25.697136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:25.697139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:25.697332 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:25.698337 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:25.698345 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:25.698596 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 00:25:25.698841 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 00:25:25.699601 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":165,"thread_start_us":454,"threads_started":1}
I20260811 00:25:25.704123 25028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:25:25.705453 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.704938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48754)
0811 00:25:25.705141 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.705295 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 00:25:25.705777 24934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
I20260811 00:25:25.707562 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.707134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48758)
0811 00:25:25.707288 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.707377 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 00:25:25.707790 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:36207: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab.
Entry for principal kudu/127.16.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab.
I20260811 00:25:25.740986 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:36207
--webserver_interface=127.16.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab
--principal=kudu/127.16.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:43973,127.16.71.189:36207,127.16.71.188:45289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:25:25.757207 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } attempt: 1
I20260811 00:25:25.758409 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.757921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48766)
0811 00:25:25.758129 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.758261 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 00:25:25.759442 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.759067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48774)
0811 00:25:25.759220 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.759292 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:25:25.759649 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:36207: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
I20260811 00:25:25.827937 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } attempt: 2
I20260811 00:25:25.829360 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.828834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48786)
0811 00:25:25.829108 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.829207 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 00:25:25.830505 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.830080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48788)
0811 00:25:25.830256 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.830353 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 00:25:25.830721 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:36207: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
I20260811 00:25:25.922026 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } attempt: 3
I20260811 00:25:25.923362 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.922847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48802)
0811 00:25:25.923089 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.923195 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 00:25:25.924438 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:25.924044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48808)
0811 00:25:25.924201 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:25.924277 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 00:25:25.924666 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:36207: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
I20260811 00:25:26.067972 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } attempt: 4
I20260811 00:25:26.069329 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.068746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48822)
0811 00:25:26.069081 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.069177 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260811 00:25:26.070685 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.070252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48838)
0811 00:25:26.070437 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.070537 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 00:25:26.070904 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.189:36207: Network error: Client connection negotiation failed: client connection to 127.16.71.189:36207: connect: Connection refused (error 111)
W20260811 00:25:26.120363 25039 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:25:26.120839 25039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:26.120942 25039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:26.121070 25039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:26.128908 25039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:26.129040 25039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:26.129136 25039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:26.129204 25039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:26.129297 25039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:26.139909 25039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:43973,127.16.71.189:36207,127.16.71.188:45289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.189.keytab
--principal=kudu/127.16.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.189:36207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:26.141434 25039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:26.142856 25039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:26.155669 25049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:26.155702 25054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:26.155741 25051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:26.156589 25039 server_base.cc:1034] running on GCE node
I20260811 00:25:26.157529 25039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:26.159317 25039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:26.160593 25039 hybrid_clock.cc:648] HybridClock initialized: now 1786407926160503 us; error 59 us; skew 500 ppm
I20260811 00:25:26.161047 25039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:26.164652 25039 init.cc:377] Logged in from keytab as kudu/127.16.71.189@KRBTEST.COM (short username kudu)
I20260811 00:25:26.166637 25039 webserver.cc:460] Webserver started at http://127.16.71.189:39831/ using document root <none> and password file <none>
I20260811 00:25:26.167532 25039 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:26.167663 25039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:26.168012 25039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:26.171355 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data/instance:
uuid: "ebd8c677b8a847bab5c28c36a6ec8e21"
format_stamp: "Formatted at 2026-08-11 00:25:26 on dist-test-slave-f7th"
server_key: "f05d299a100d2558fdced7d4de7dcfc0"
server_key_iv: "0cc7d8fd7c913692f922c061258df9bc"
server_key_version: "encryptionkey@0"
I20260811 00:25:26.172269 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/wal/instance:
uuid: "ebd8c677b8a847bab5c28c36a6ec8e21"
format_stamp: "Formatted at 2026-08-11 00:25:26 on dist-test-slave-f7th"
server_key: "f05d299a100d2558fdced7d4de7dcfc0"
server_key_iv: "0cc7d8fd7c913692f922c061258df9bc"
server_key_version: "encryptionkey@0"
I20260811 00:25:26.178375 25039 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 00:25:26.182368 25062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:26.184012 25039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:25:26.184217 25039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/wal
uuid: "ebd8c677b8a847bab5c28c36a6ec8e21"
format_stamp: "Formatted at 2026-08-11 00:25:26 on dist-test-slave-f7th"
server_key: "f05d299a100d2558fdced7d4de7dcfc0"
server_key_iv: "0cc7d8fd7c913692f922c061258df9bc"
server_key_version: "encryptionkey@0"
I20260811 00:25:26.184449 25039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:26.207898 25039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:26.212653 25039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:26.213002 25039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:26.233299 25039 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.189:36207
I20260811 00:25:26.233323 25151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.189:36207 every 8 connection(s)
I20260811 00:25:26.234836 25039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/data/info.pb
I20260811 00:25:26.237949 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 25039
I20260811 00:25:26.238147 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-1/wal/instance
I20260811 00:25:26.238616 16670 external_mini_cluster.cc:1428] Setting key da7703b03a270f72d7e4fdfef457e5ea
I20260811 00:25:26.239262 25153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:26.246703 25153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:25:26.270789 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.249752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.16.71.190:43973)
0811 00:25:26.250262 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.250266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.251533 (+ 1267us) server_negotiation.cc:407] Connection header received
0811 00:25:26.254071 (+ 2538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.254076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.254170 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.254272 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.256174 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.256980 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.258728 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.259171 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.259297 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.263936 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.263943 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.263947 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.264003 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.266950 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.267989 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.267995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.267999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.268187 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.268684 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.268690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.268693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.269117 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.269291 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.270091 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.270361 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":273,"threads_started":1}
WARNING: no policy specified for kudu/127.16.71.188@KRBTEST.COM; defaulting to no policy
I20260811 00:25:26.271667 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.250445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:43973 (local address 127.0.0.1:57080)
0811 00:25:26.251342 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.251387 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:26.253703 (+ 2316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:26.254600 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:26.254611 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:26.255219 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:26.255879 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.255901 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.257185 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.257190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.258503 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.258537 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.259460 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.259470 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.259725 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.260655 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:26.260695 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:26.263569 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:26.267184 (+ 3615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.267199 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.267224 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.267822 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.268318 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.268321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.268323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.268542 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.269246 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:26.269253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:26.269664 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 00:25:26.269932 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 00:25:26.270432 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":347,"threads_started":1}
I20260811 00:25:26.273972 25153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.71.188@KRBTEST.COM" created.
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:25:26.290982 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.274940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48854)
0811 00:25:26.275150 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.275171 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:26.275324 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:26.276391 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:26.276396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:26.276905 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:26.277063 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.277073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.279023 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.279028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.279910 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.279923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.281108 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.281113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.281264 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.281882 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:26.281899 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:26.284310 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:26.287903 (+ 3593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.287909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.287913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.288482 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.288948 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.288955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.288958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.289120 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.289930 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:26.289935 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:26.290084 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 00:25:26.290208 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 00:25:26.290577 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 00:25:26.291139 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.275054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.16.71.189:36207)
0811 00:25:26.275740 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.275749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.275785 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:25:26.275867 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.275876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.276167 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.276337 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.277260 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.278797 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.280076 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.280955 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.281080 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.284583 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.284599 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.284645 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.284699 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.287710 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.288677 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.288686 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.288688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.288826 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.289227 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.289232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.289235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.289784 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.289993 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.290393 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.290728 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":488,"thread_start_us":300,"threads_started":1}
I20260811 00:25:26.293843 25153 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:25:26.295195 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.294627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52568)
0811 00:25:26.294867 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.295019 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 00:25:26.295511 25063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.297289 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.296914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52578)
0811 00:25:26.297065 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.297160 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:25:26.297696 25153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab.
Entry for principal kudu/127.16.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab.
I20260811 00:25:26.320609 16670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9TQxI2/build/asan/bin/kudu
/tmp/dist-test-task9TQxI2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:45289
--webserver_interface=127.16.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab
--principal=kudu/127.16.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.71.148:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.71.190:43973,127.16.71.189:36207,127.16.71.188:45289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:25:26.347097 25153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 1
I20260811 00:25:26.348668 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.348060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52580)
0811 00:25:26.348320 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.348446 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 00:25:26.350066 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.349556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52586)
0811 00:25:26.349775 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.349891 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:25:26.350339 25153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.370206 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } attempt: 5
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:25:26.385846 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.371258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48856 (local address 127.16.71.189:36207)
0811 00:25:26.371480 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.371487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.371506 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:25:26.371583 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.371587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.371650 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.371736 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.372742 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.373834 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.375152 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.375868 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.376021 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.379325 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.379332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.379337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.379405 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.382842 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.383857 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.383862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.383864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.384004 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.384488 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.384491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.384493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.384861 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.384983 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.385429 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.385621 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:25:26.386036 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.371025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48856)
0811 00:25:26.371218 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.371238 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:26.371366 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:26.371798 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:26.371803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:26.372239 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:26.372499 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.372517 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.374024 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.374029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.374962 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.374977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.376059 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.376065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.376254 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.376924 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:26.376943 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:26.379047 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:26.383075 (+ 4028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.383081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.383085 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.383680 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.384160 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.384165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.384169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.384353 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.384984 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:26.384989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:26.385175 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 00:25:26.385343 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 00:25:26.385701 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 00:25:26.387252 25028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:25:26.388411 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.387844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52602)
0811 00:25:26.388086 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.388202 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 00:25:26.389596 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.389141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52612)
0811 00:25:26.389315 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.389446 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:25:26.389954 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.418637 25153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 2
I20260811 00:25:26.420013 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.419420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52622)
0811 00:25:26.419686 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.419811 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 00:25:26.421295 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.420850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52630)
0811 00:25:26.421018 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.421117 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 00:25:26.421569 25153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.441224 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 1
I20260811 00:25:26.442757 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.442015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52634)
0811 00:25:26.442360 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.442506 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 00:25:26.443994 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.443565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52650)
0811 00:25:26.443751 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.443850 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 00:25:26.444258 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.512552 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 2
I20260811 00:25:26.512817 25153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 3
I20260811 00:25:26.514122 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.513490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52664)
0811 00:25:26.513758 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.513874 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 00:25:26.514127 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.513635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52678)
0811 00:25:26.513864 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.513960 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:25:26.515475 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.514971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52682)
0811 00:25:26.515268 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.515351 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260811 00:25:26.515516 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.514972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52680)
0811 00:25:26.515175 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.515289 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 00:25:26.515753 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
W20260811 00:25:26.515839 25153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.622056 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 3
I20260811 00:25:26.623374 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.622812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52710)
0811 00:25:26.623095 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.623195 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 00:25:26.624568 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.624102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52720)
0811 00:25:26.624347 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.624441 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 00:25:26.624933 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.659082 25153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 4
I20260811 00:25:26.660315 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.659788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52732)
0811 00:25:26.660049 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.660140 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 00:25:26.661433 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.661062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52746)
0811 00:25:26.661224 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.661306 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 00:25:26.661696 25153 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
W20260811 00:25:26.696909 25163 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:25:26.697278 25163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:26.697350 25163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:26.697432 25163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:26.705367 25163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:26.705497 25163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:26.705557 25163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:26.705610 25163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:26.705672 25163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:26.716264 25163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.71.148:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.71.190:43973,127.16.71.189:36207,127.16.71.188:45289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/krb5kdc/kudu_127.16.71.188.keytab
--principal=kudu/127.16.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.71.188:45289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:26.717731 25163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:26.719123 25163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:26.732035 25169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:26.732013 25175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:26.731961 25171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:26.732869 25163 server_base.cc:1034] running on GCE node
I20260811 00:25:26.733752 25163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:26.735386 25163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:26.736723 25163 hybrid_clock.cc:648] HybridClock initialized: now 1786407926736626 us; error 55 us; skew 500 ppm
I20260811 00:25:26.737156 25163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:26.740657 25163 init.cc:377] Logged in from keytab as kudu/127.16.71.188@KRBTEST.COM (short username kudu)
I20260811 00:25:26.742487 25163 webserver.cc:460] Webserver started at http://127.16.71.188:37965/ using document root <none> and password file <none>
I20260811 00:25:26.743306 25163 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:26.743430 25163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:26.743774 25163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:26.747256 25163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data/instance:
uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0"
format_stamp: "Formatted at 2026-08-11 00:25:26 on dist-test-slave-f7th"
server_key: "59323cb2d1e15ff7509d0ff24450be81"
server_key_iv: "7b8aab15cbaa1d074409f6e6202d7259"
server_key_version: "encryptionkey@0"
I20260811 00:25:26.748164 25163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/wal/instance:
uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0"
format_stamp: "Formatted at 2026-08-11 00:25:26 on dist-test-slave-f7th"
server_key: "59323cb2d1e15ff7509d0ff24450be81"
server_key_iv: "7b8aab15cbaa1d074409f6e6202d7259"
server_key_version: "encryptionkey@0"
I20260811 00:25:26.754859 25163 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260811 00:25:26.759310 25188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:26.761215 25163 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:26.761468 25163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data,/tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/wal
uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0"
format_stamp: "Formatted at 2026-08-11 00:25:26 on dist-test-slave-f7th"
server_key: "59323cb2d1e15ff7509d0ff24450be81"
server_key_iv: "7b8aab15cbaa1d074409f6e6202d7259"
server_key_version: "encryptionkey@0"
I20260811 00:25:26.761731 25163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:26.782728 25163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:26.787703 25163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:26.788002 25163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:26.802175 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 4
I20260811 00:25:26.803826 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.802993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52754)
0811 00:25:26.803323 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.803585 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 00:25:26.805142 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.804686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52758)
0811 00:25:26.804854 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.804956 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 00:25:26.805421 25028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.71.188:45289: Network error: Client connection negotiation failed: client connection to 127.16.71.188:45289: connect: Connection refused (error 111)
I20260811 00:25:26.813109 25163 rpc_server.cc:307] RPC server started. Bound to: 127.16.71.188:45289
I20260811 00:25:26.813130 25276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.71.188:45289 every 8 connection(s)
I20260811 00:25:26.814852 25163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/data/info.pb
I20260811 00:25:26.817404 16670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9TQxI2/build/asan/bin/kudu as pid 25163
I20260811 00:25:26.817595 16670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9TQxI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407527335556-16670-0/minicluster-data/master-2/wal/instance
I20260811 00:25:26.818038 16670 external_mini_cluster.cc:1428] Setting key 73181698fbcb75dd7ab725d86e7a94ab
I20260811 00:25:26.820132 25277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:26.829676 25277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
I20260811 00:25:26.845968 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.820176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.16.71.188:45289)
0811 00:25:26.821258 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.821272 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.821318 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:25:26.822986 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.823013 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.823497 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.823878 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.826018 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.827584 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.829512 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.830298 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.830497 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.835460 (+ 4963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.835481 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.835496 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.835549 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.840084 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.841408 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.841422 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.841434 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.841728 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.842301 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.842308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.842310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.843106 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.843384 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.844073 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.844603 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":543,"threads_started":1}
I20260811 00:25:26.852552 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.832957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.190:43973 (local address 127.0.0.1:57084)
0811 00:25:26.833994 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.834041 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:26.834230 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:26.834657 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:26.834663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:26.835477 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:26.835900 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.835919 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.837222 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.837228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.839625 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.839650 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.841241 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.841247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.841444 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.842286 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:26.842307 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:26.845303 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:26.849419 (+ 4116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.849435 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.849442 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.850116 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.850570 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.850574 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.850577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.850733 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.851320 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:26.851329 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:26.851542 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 00:25:26.851720 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 00:25:26.852104 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":567,"threads_started":1}
I20260811 00:25:26.852557 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.832956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57084 (local address 127.16.71.190:43973)
0811 00:25:26.833574 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.833581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.834273 (+ 692us) server_negotiation.cc:407] Connection header received
0811 00:25:26.834358 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.834363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.834468 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.834591 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.836139 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.837027 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.839998 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.841008 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.841204 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.845586 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.845594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.845598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.845645 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.849255 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.850264 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.850268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.850271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.850430 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.850851 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.850855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.850858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.851166 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.851308 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.851867 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.852130 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":262,"threads_started":1}
I20260811 00:25:26.854302 25277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.190@KRBTEST.COM
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
I20260811 00:25:26.866194 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.849570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.16.71.190:43973)
0811 00:25:26.850224 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.850228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.850248 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:25:26.850293 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.850296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.850358 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.850456 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.851801 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.852804 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.854442 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.855073 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.855206 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.859769 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.859776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.859780 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.859825 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.863061 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.863913 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.863917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.863920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.864077 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.864500 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.864504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.864506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.864888 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.865163 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.865519 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.865773 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":303,"threads_started":1}
I20260811 00:25:26.869122 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.855246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.189:36207 (local address 127.0.0.1:48862)
0811 00:25:26.855517 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.855533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:26.855625 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:26.855951 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:26.855956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:26.856354 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:26.856504 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.856512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.857496 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.857502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.858702 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.858721 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.859460 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.859466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.859671 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.860276 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:26.860307 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:26.862621 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:26.865853 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.865859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.865864 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.866598 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.867177 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.867183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.867186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.867391 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.868012 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:26.868016 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:26.868181 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 00:25:26.868343 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 00:25:26.868713 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 00:25:26.869982 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.855189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.16.71.189:36207)
0811 00:25:26.855394 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.855400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.855607 (+ 207us) server_negotiation.cc:407] Connection header received
0811 00:25:26.855718 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.855723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.855806 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.855895 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.856680 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.857356 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.858873 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.859296 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.859415 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.862855 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.862861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.862864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.862925 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.865700 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.866789 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.866794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.866797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.867032 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.867539 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.867545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.867548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.867878 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.868046 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.869463 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.869676 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:25:26.871181 25277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.71.189@KRBTEST.COM
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.188@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
I20260811 00:25:26.885741 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.871855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52780)
0811 00:25:26.872016 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.872045 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:26.872155 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:26.872434 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:26.872438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:26.872910 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:26.873062 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.873071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.874037 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.874042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.875311 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.875330 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.876390 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.876395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.876575 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.877203 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:26.877222 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:26.879714 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:26.882984 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.882990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.882994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.883433 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.883908 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.883912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.883914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.884048 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.884601 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:26.884606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:26.884794 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 00:25:26.884942 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 00:25:26.885305 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":23}
I20260811 00:25:26.885805 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.871974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.16.71.188:45289)
0811 00:25:26.872117 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.872123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.872140 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:25:26.872257 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.872262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.872325 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.872400 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.873191 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.873903 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.875505 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.876185 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.876335 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.879921 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.879926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.879929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.879963 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.882790 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.883568 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.883574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.883577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.883720 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.884161 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.884165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.884167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.884478 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.884683 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.885049 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.885490 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":49}
I20260811 00:25:26.886385 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.868446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.16.71.189:36207)
0811 00:25:26.869333 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.869340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.869370 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:25:26.869426 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.869430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.869506 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.869620 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.871303 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.872298 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.873736 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.874219 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.874345 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.879956 (+ 5611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.879962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.879966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.880018 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.882984 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.883835 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.883840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.883842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.884049 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.884609 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.884642 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.884646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.885043 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.885340 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.885706 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.885968 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":499,"threads_started":1}
I20260811 00:25:26.893136 25277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: Bootstrap starting.
I20260811 00:25:26.893394 16670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:25:26.897301 25277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:26.898909 25277 log.cc:824] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:26.896065 24979 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6c70aed3cdb340ada4e6f86f4857e72a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57096
I20260811 00:25:26.902812 25277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: No bootstrap required, opened a new log
I20260811 00:25:26.914295 25277 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
I20260811 00:25:26.914856 25277 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ad11c380ff46c8a2b02d0e0b2b38d0, State: Initialized, Role: FOLLOWER
I20260811 00:25:26.915465 25277 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
I20260811 00:25:26.916772 25292 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:26.917215 25292 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:26.918367 25277 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:25:26.937463 25309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:25:26.937693 25309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:25:26.934966 25163 master_runner.cc:186] Error getting master registration for 127.16.71.190:43973: OK, instance_id { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" instance_seqno: 1786407925649957 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:25:26.946722 25163 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:25:26.960922 25153 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 5
Aug 11 00:25:26 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.189@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
I20260811 00:25:26.978734 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.961701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52796)
0811 00:25:26.961916 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:26.961939 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:26.962119 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:26.962506 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:26.962510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:26.962954 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:26.963153 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.963166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.964706 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.964710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.965779 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:26.965794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.966902 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.966909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:26.967191 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.967882 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:26.967904 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:26.970926 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:26.975291 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.975298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.975303 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.976077 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.976608 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:26.976637 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:26.976641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:26.976821 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:26.977634 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:26.977640 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:26.977820 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 00:25:26.977982 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 00:25:26.978310 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 00:25:26.978762 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:26.961794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.16.71.188:45289)
0811 00:25:26.961992 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:26.961998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:26.962019 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:25:26.962263 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:26.962269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:26.962350 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:26.962442 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:26.963336 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.964486 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.965928 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:26.966726 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:26.966881 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:26.971226 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:26.971235 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:26.971240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:26.971291 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:26.975061 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.976263 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.976270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.976274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.976471 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:26.976953 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:26.976959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:26.976962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:26.977491 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:26.977671 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:26.978136 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 00:25:26.978383 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:25:26.983841 25153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: Bootstrap starting.
I20260811 00:25:26.988305 25153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:26.989658 25153 log.cc:824] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:26.992880 25153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: No bootstrap required, opened a new log
I20260811 00:25:27.001150 25153 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
I20260811 00:25:27.001566 25153 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd8c677b8a847bab5c28c36a6ec8e21, State: Initialized, Role: FOLLOWER
I20260811 00:25:27.002104 25153 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
I20260811 00:25:27.003561 25312 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.004011 25312 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:27.005002 25153 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:25:27.016431 25336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:25:27.016664 25336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:25:27.013564 25039 master_runner.cc:186] Error getting master registration for 127.16.71.190:43973: OK, instance_id { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" instance_seqno: 1786407925649957 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:25:27.023840 25039 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 00:25:27.040760 25196 tablet.cc:2369] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:27.082687 25028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } attempt: 5
Aug 11 00:25:27 dist-test-slave-f7th krb5kdc[24882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.71.190@KRBTEST.COM for kudu/127.16.71.188@KRBTEST.COM
I20260811 00:25:27.102342 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:27.083538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.71.188:45289 (local address 127.0.0.1:52812)
0811 00:25:27.083873 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:27.083909 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:27.084072 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:27.084792 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:27.084799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:27.085370 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:27.085626 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:27.085642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.087285 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.087292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:27.088945 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:27.088980 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.090139 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.090146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:27.090376 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:27.091125 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:27.091154 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:27.094288 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:27.099126 (+ 4838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:27.099134 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:27.099139 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:27.099771 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:27.100280 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:27.100285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:27.100289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:27.100502 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:27.101196 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:27.101202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:27.101401 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 00:25:27.101554 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 00:25:27.101886 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260811 00:25:27.103454 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:27.083635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.16.71.188:45289)
0811 00:25:27.083918 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.083925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.084008 (+ 83us) server_negotiation.cc:407] Connection header received
0811 00:25:27.084247 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.084252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.084349 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.084478 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.085898 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.087112 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.089158 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.089956 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.090099 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:27.094574 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:27.094582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:27.094587 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:27.094638 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:27.098895 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.099942 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.099948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.099952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.100149 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.100655 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.100660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.100664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.101057 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:27.101221 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:27.102792 (+ 1571us) server_negotiation.cc:299] Negotiation successful
0811 00:25:27.103072 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 00:25:27.110401 25028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a: Bootstrap starting.
I20260811 00:25:27.115160 25028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:27.116807 25028 log.cc:824] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:27.122020 25028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a: No bootstrap required, opened a new log
I20260811 00:25:27.131120 25028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
I20260811 00:25:27.131647 25028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c70aed3cdb340ada4e6f86f4857e72a, State: Initialized, Role: FOLLOWER
I20260811 00:25:27.132247 25028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
I20260811 00:25:27.133734 25339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.134600 25339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:27.135082 25028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:27.135161 24979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 00:25:27.135378 24979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:27.139256 24979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
W20260811 00:25:27.139931 24940 tablet.cc:2369] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:27.142612 25111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21"
I20260811 00:25:27.143311 25111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:27.147306 25111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c70aed3cdb340ada4e6f86f4857e72a in term 1.
I20260811 00:25:27.148386 24934 leader_election.cc:302] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c70aed3cdb340ada4e6f86f4857e72a, ebd8c677b8a847bab5c28c36a6ec8e21; no voters:
I20260811 00:25:27.150367 25233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0"
I20260811 00:25:27.151185 25233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:27.155503 25233 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c70aed3cdb340ada4e6f86f4857e72a in term 1.
I20260811 00:25:27.156817 24979 leader_election.cc:288] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [CANDIDATE]: Term 1 election: Requested vote from peers ebd8c677b8a847bab5c28c36a6ec8e21 (127.16.71.189:36207), a9ad11c380ff46c8a2b02d0e0b2b38d0 (127.16.71.188:45289)
I20260811 00:25:27.164072 25339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:27.177083 25339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [term 1 LEADER]: Becoming Leader. State: Replica: 6c70aed3cdb340ada4e6f86f4857e72a, State: Running, Role: LEADER
I20260811 00:25:27.177999 25339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } }
I20260811 00:25:27.189131 25347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c70aed3cdb340ada4e6f86f4857e72a. Latest consensus state: current_term: 1 leader_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.189828 25347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:27.195135 25360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 00:25:27.216037 25072 tablet.cc:2369] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:27.219319 25360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:27.248060 25374 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:25:27.258383 25233 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [term 1 FOLLOWER]: Refusing update from remote peer 6c70aed3cdb340ada4e6f86f4857e72a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:27.259178 25111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [term 1 FOLLOWER]: Refusing update from remote peer 6c70aed3cdb340ada4e6f86f4857e72a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:27.259907 25347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:27.260726 25339 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:27.266917 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:27.200038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57100 (local address 127.16.71.190:43973)
0811 00:25:27.200240 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.200247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.213034 (+ 12787us) server_negotiation.cc:407] Connection header received
0811 00:25:27.213958 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.213964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.214067 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.214163 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.224060 (+ 9897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.225186 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.228719 (+ 3533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.229795 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.229952 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:27.241971 (+ 12019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:27.241979 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:27.241985 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:27.242047 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:27.246481 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.249237 (+ 2756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.249269 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.249273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.249565 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.253772 (+ 4207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.253779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.253783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.254259 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:27.254528 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:27.260230 (+ 5702us) server_negotiation.cc:299] Negotiation successful
0811 00:25:27.260566 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 00:25:27.287746 25292 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c70aed3cdb340ada4e6f86f4857e72a. Latest consensus state: current_term: 1 leader_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.288146 25292 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:27.290277 25375 mvcc.cc:205] Tried to move back new op lower bound from 7317126869977014272 to 7317126869740441600. Current Snapshot: MvccSnapshot[applied={T|T < 7317126869977014272}]
I20260811 00:25:27.293355 25312 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c70aed3cdb340ada4e6f86f4857e72a. Latest consensus state: current_term: 1 leader_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.293766 25312 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:27.307375 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:27.199218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.16.71.189:36207)
0811 00:25:27.199715 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.199722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.213571 (+ 13849us) server_negotiation.cc:407] Connection header received
0811 00:25:27.216788 (+ 3217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.216794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.216897 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.216992 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.219721 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.220782 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.247385 (+ 26603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.248078 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.248203 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:27.281150 (+ 32947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:27.281159 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:27.281165 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:27.281224 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:27.285560 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.286864 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.286871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.286875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.287075 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.287655 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.287661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.287664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.288145 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:27.288503 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:27.306632 (+ 18129us) server_negotiation.cc:299] Negotiation successful
0811 00:25:27.306993 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319}
I20260811 00:25:27.319151 25292 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.319286 25347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.319478 25292 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:27.319558 25347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:27.320420 25347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.320788 25347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:27.324293 25360 catalog_manager.cc:1347] Generated new cluster ID: 3aafe65263e746cb879e5191b97d0e62
I20260811 00:25:27.324419 25360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:27.324756 25312 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c70aed3cdb340ada4e6f86f4857e72a" member_type: VOTER last_known_addr { host: "127.16.71.190" port: 43973 } } peers { permanent_uuid: "ebd8c677b8a847bab5c28c36a6ec8e21" member_type: VOTER last_known_addr { host: "127.16.71.189" port: 36207 } } peers { permanent_uuid: "a9ad11c380ff46c8a2b02d0e0b2b38d0" member_type: VOTER last_known_addr { host: "127.16.71.188" port: 45289 } } }
I20260811 00:25:27.325042 25312 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:25:27.325603 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:27.214253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.16.71.188:45289)
0811 00:25:27.214520 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.214526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.219597 (+ 5071us) server_negotiation.cc:407] Connection header received
0811 00:25:27.244835 (+ 25238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.244845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.244995 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.245093 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.249441 (+ 4348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.251456 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.273568 (+ 22112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:27.274385 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:27.274532 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:27.309136 (+ 34604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:27.309143 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:27.309148 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:27.309200 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:27.313770 (+ 4570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.315096 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.315104 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.315108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.315308 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:27.317231 (+ 1923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:27.317239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:27.317243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:27.317886 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:27.318200 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:27.324915 (+ 6715us) server_negotiation.cc:299] Negotiation successful
0811 00:25:27.325246 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 00:25:27.370379 25360 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:27.376171 25360 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:27.402889 25360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c70aed3cdb340ada4e6f86f4857e72a: Generated new TSK 0
I20260811 00:25:27.404153 25360 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:25:27.531814 25286 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.523107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.16.71.190:43973)
0811 00:25:27.523311 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.523317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.526995 (+ 3678us) server_negotiation.cc:407] Connection header received
0811 00:25:27.529825 (+ 2830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.529832 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.529950 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.530060 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.531435 (+ 1375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57110: BlockingRecv error: recv got EOF from 127.0.0.1:57110 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 00:25:27.534515 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.522656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.16.71.189:36207)
0811 00:25:27.522923 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.522929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.529672 (+ 6743us) server_negotiation.cc:407] Connection header received
0811 00:25:27.532679 (+ 3007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.532687 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.532786 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.532863 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.534244 (+ 1381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48888: BlockingRecv error: recv got EOF from 127.0.0.1:48888 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 00:25:27.538013 25281 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.527236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.16.71.188:45289)
0811 00:25:27.527481 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.527488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.535500 (+ 8012us) server_negotiation.cc:407] Connection header received
0811 00:25:27.536162 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.536168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.536280 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.536363 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.537693 (+ 1330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52826: BlockingRecv error: recv got EOF from 127.0.0.1:52826 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260811 00:25:27.553153 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.550535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.16.71.189:36207)
0811 00:25:27.550745 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.550753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.551347 (+ 594us) server_negotiation.cc:407] Connection header received
0811 00:25:27.551639 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.551645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.551816 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.551944 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.552885 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48898: BlockingRecv error: recv got EOF from 127.0.0.1:48898 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 00:25:27.560771 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.558558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.16.71.189:36207)
0811 00:25:27.558765 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.558772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.559088 (+ 316us) server_negotiation.cc:407] Connection header received
0811 00:25:27.559302 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.559308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.559406 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.559522 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.560433 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48902: BlockingRecv error: recv got EOF from 127.0.0.1:48902 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 00:25:27.569319 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.566781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.16.71.189:36207)
0811 00:25:27.567020 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.567026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.567611 (+ 585us) server_negotiation.cc:407] Connection header received
0811 00:25:27.567841 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.567847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.567949 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.568077 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.569045 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48904: BlockingRecv error: recv got EOF from 127.0.0.1:48904 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 00:25:27.581423 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.578910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.16.71.189:36207)
0811 00:25:27.579162 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.579172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.579702 (+ 530us) server_negotiation.cc:407] Connection header received
0811 00:25:27.579906 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.579912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.580043 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.580173 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.581157 (+ 984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48916: BlockingRecv error: recv got EOF from 127.0.0.1:48916 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260811 00:25:27.595806 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.593112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.16.71.189:36207)
0811 00:25:27.593315 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.593321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.594037 (+ 716us) server_negotiation.cc:407] Connection header received
0811 00:25:27.594366 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.594373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.594499 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.594616 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.595577 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48926: BlockingRecv error: recv got EOF from 127.0.0.1:48926 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 00:25:27.618760 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.615979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.16.71.189:36207)
0811 00:25:27.616239 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.616245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.616926 (+ 681us) server_negotiation.cc:407] Connection header received
0811 00:25:27.617194 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.617202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.617303 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.617408 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.618429 (+ 1021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48942: BlockingRecv error: recv got EOF from 127.0.0.1:48942 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 00:25:27.656960 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.654664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.16.71.189:36207)
0811 00:25:27.654888 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.654914 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.655305 (+ 391us) server_negotiation.cc:407] Connection header received
0811 00:25:27.655549 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.655555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.655655 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.655762 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.656727 (+ 965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48948: BlockingRecv error: recv got EOF from 127.0.0.1:48948 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 00:25:27.726910 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.724701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.16.71.189:36207)
0811 00:25:27.724959 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.724965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.725350 (+ 385us) server_negotiation.cc:407] Connection header received
0811 00:25:27.725563 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.725568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.725667 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.725757 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.726646 (+ 889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48960: BlockingRecv error: recv got EOF from 127.0.0.1:48960 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 00:25:27.862146 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:25:27.859815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.16.71.189:36207)
0811 00:25:27.860035 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:27.860040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:27.860521 (+ 481us) server_negotiation.cc:407] Connection header received
0811 00:25:27.860921 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:27.860926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:27.861013 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:27.861122 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:27.861903 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48976: BlockingRecv error: recv got EOF from 127.0.0.1:48976 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:25:27.939872 25309 catalog_manager.cc:1259] Loaded cluster ID: 3aafe65263e746cb879e5191b97d0e62
I20260811 00:25:27.939992 25309 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: loading cluster ID for follower catalog manager: success
I20260811 00:25:27.942746 25309 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: acquiring CA information for follower catalog manager: success
I20260811 00:25:27.945185 25309 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9ad11c380ff46c8a2b02d0e0b2b38d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:25:28.019012 25336 catalog_manager.cc:1259] Loaded cluster ID: 3aafe65263e746cb879e5191b97d0e62
I20260811 00:25:28.019129 25336 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: loading cluster ID for follower catalog manager: success
I20260811 00:25:28.021970 25336 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: acquiring CA information for follower catalog manager: success
I20260811 00:25:28.024025 25336 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ebd8c677b8a847bab5c28c36a6ec8e21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:25:28.128286 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:28.122027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.16.71.189:36207)
0811 00:25:28.122266 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:28.122273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:28.122715 (+ 442us) server_negotiation.cc:407] Connection header received
0811 00:25:28.122947 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:28.122951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:28.123140 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:28.123252 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:28.123741 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.124495 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.125930 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.126347 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.126469 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:28.127038 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:28.127607 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:28.127753 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:28.127872 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 00:25:28.127971 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 00:25:28.141384 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:28.134793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52842 (local address 127.16.71.188:45289)
0811 00:25:28.135273 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:28.135278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:28.135373 (+ 95us) server_negotiation.cc:407] Connection header received
0811 00:25:28.135660 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:28.135665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:28.135837 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:28.136012 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:28.136557 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.137336 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.138837 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.139256 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.139369 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:28.139975 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:28.140605 (+ 630us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:28.140786 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:28.140949 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 00:25:28.141061 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":245,"threads_started":1}
I20260811 00:25:28.156345 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:28.149645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.16.71.188:45289)
0811 00:25:28.150180 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:28.150186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:28.150332 (+ 146us) server_negotiation.cc:407] Connection header received
0811 00:25:28.150732 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:28.150739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:28.150830 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:28.150934 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:28.151500 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.152438 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.154383 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.154826 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.154942 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:28.155521 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:28.155667 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:28.155752 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:28.155982 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 00:25:28.156068 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397}
I20260811 00:25:28.156538 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:28.148006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48984 (local address 127.16.71.189:36207)
0811 00:25:28.148272 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:28.148279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:28.149196 (+ 917us) server_negotiation.cc:407] Connection header received
0811 00:25:28.149705 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:28.149712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:28.149831 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:28.149898 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:28.150885 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.151805 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.154248 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.154937 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.155036 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:28.155736 (+ 700us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:28.155862 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:28.155948 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:28.156205 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 00:25:28.156288 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:25:28.157269 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:28.147971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.16.71.190:43973)
0811 00:25:28.149606 (+ 1635us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:28.149613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:28.149637 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:25:28.149767 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:28.149772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:28.150019 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:28.150258 (+ 239us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:28.151058 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.152041 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.154248 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:28.154884 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:28.155310 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:28.155732 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:28.156286 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:28.156855 (+ 569us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:28.156907 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 00:25:28.157001 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1387,"thread_start_us":1738,"threads_started":1}
I20260811 00:25:28.163545 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 24904
I20260811 00:25:28.164798 25016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:28.276399 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 24904
I20260811 00:25:28.311262 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 25039
I20260811 00:25:28.312677 25146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:28.425135 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 25039
I20260811 00:25:28.455894 16670 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9TQxI2/build/asan/bin/kudu pid 25163
I20260811 00:25:28.457350 25270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:28.566998 16670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9TQxI2/build/asan/bin/kudu with pid 25163
2026-08-11T00:25:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5118 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10206 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (401170 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST